diff --git a/.gitattributes b/.gitattributes index c7e0c4779df108cca06ce19a3019c16992a5df0d..86a861a820f7108ce39f6eb66320bb5e8b9e3a06 100644 --- a/.gitattributes +++ b/.gitattributes @@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text *tfevents* filter=lfs diff=lfs merge=lfs -text git.diff filter=lfs diff=lfs merge=lfs -text replay.mp4 filter=lfs diff=lfs merge=lfs -text +sf_log.txt filter=lfs diff=lfs merge=lfs -text diff --git a/.summary/0/events.out.tfevents.1701168931.rhmmedcatt-proliant-ml350-gen10 b/.summary/0/events.out.tfevents.1701168931.rhmmedcatt-proliant-ml350-gen10 new file mode 100644 index 0000000000000000000000000000000000000000..c53757fce8815c23dfe8766e9d1e424837799f20 --- /dev/null +++ b/.summary/0/events.out.tfevents.1701168931.rhmmedcatt-proliant-ml350-gen10 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5162d703f55352d9d36a961096fd15c1632486a442989f2b22a9b55555056e0a +size 40 diff --git a/.summary/0/events.out.tfevents.1701404001.rhmmedcatt-proliant-ml350-gen10 b/.summary/0/events.out.tfevents.1701404001.rhmmedcatt-proliant-ml350-gen10 new file mode 100644 index 0000000000000000000000000000000000000000..d405caa03c412e8fd677c2a455ec70e5c1e76755 --- /dev/null +++ b/.summary/0/events.out.tfevents.1701404001.rhmmedcatt-proliant-ml350-gen10 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32ebeaedcc62c342764367a7bdba5eabaab07ef1590733651828c24828cef1ff +size 354182 diff --git a/.summary/0/events.out.tfevents.1701587986.rhmmedcatt-proliant-ml350-gen10 b/.summary/0/events.out.tfevents.1701587986.rhmmedcatt-proliant-ml350-gen10 new file mode 100644 index 0000000000000000000000000000000000000000..2b951780991773423c442ed4280ceeb4f44fb87d --- /dev/null +++ b/.summary/0/events.out.tfevents.1701587986.rhmmedcatt-proliant-ml350-gen10 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dccb2eefdee7e0755933a0f4f2a7f5297af93fa128334d25eb6797a8bc72e932 +size 83938711 diff --git a/.summary/1/events.out.tfevents.1701168931.rhmmedcatt-proliant-ml350-gen10 b/.summary/1/events.out.tfevents.1701168931.rhmmedcatt-proliant-ml350-gen10 new file mode 100644 index 0000000000000000000000000000000000000000..89e0ff7a20685c24b9d75ea0f51c43f337906a5b --- /dev/null +++ b/.summary/1/events.out.tfevents.1701168931.rhmmedcatt-proliant-ml350-gen10 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:07232d72758821aacaf9b81172babc9a44b411be34f5b9d3f7b47a27c15b3f40 +size 40 diff --git a/.summary/1/events.out.tfevents.1701404001.rhmmedcatt-proliant-ml350-gen10 b/.summary/1/events.out.tfevents.1701404001.rhmmedcatt-proliant-ml350-gen10 new file mode 100644 index 0000000000000000000000000000000000000000..e4675705262c3c979e9f023f255db6c753993fc6 --- /dev/null +++ b/.summary/1/events.out.tfevents.1701404001.rhmmedcatt-proliant-ml350-gen10 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c37ae241001c98fe4d3473ee269324ea8791bb1f70220e1829b3b6ce91ae5939 +size 246940 diff --git a/.summary/1/events.out.tfevents.1701587986.rhmmedcatt-proliant-ml350-gen10 b/.summary/1/events.out.tfevents.1701587986.rhmmedcatt-proliant-ml350-gen10 new file mode 100644 index 0000000000000000000000000000000000000000..facdf7fa8be7a061508aa6a41b7f7e8b3ad5158b --- /dev/null +++ b/.summary/1/events.out.tfevents.1701587986.rhmmedcatt-proliant-ml350-gen10 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ffff4936599bab4baf89189c5043a60c88b04980ace0e843766957f733e89ba6 +size 44033895 diff --git a/README.md b/README.md index 06ec314733a801f9f328f2c66bfd0c0a2579a602..4a596199da718e9b46cbcfb6d3f12e6d723351c4 100644 --- a/README.md +++ b/README.md @@ -15,35 +15,39 @@ model-index: type: atari_spaceinvaders metrics: - type: mean_reward - value: 2642.00 +/- 352.66 + value: 40227.50 +/- 20536.35 name: mean_reward verified: false --- -A(n) **APPO** model trained on the **atari_spaceinvaders** environment. +## About the Project -This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. -Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/ +This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). +In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this. -## Downloading the model +## Project Aims -After installing Sample-Factory, download the model with: -``` -python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_spaceinvaders -``` +This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. - -## About the Model +I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. + +The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. -This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. +After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234') -The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. -The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his. -I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes: +## About the Model + +The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency: ``` hyperparameters = { + "help": false, + "algo": "APPO", + "env": "atari_asteroid", + "experiment": "atari_asteroid_APPO", + "train_dir": "./train_atari", + "restart_behavior": "restart", "device": "gpu", "seed": 1234, "num_policies": 2, @@ -141,12 +145,28 @@ hyperparameters = { "env_gpu_observations": true, "env_frameskip": 4, "env_framestack": 4, - } + "pixel_format": "CHW" +} ``` +A(n) **APPO** model trained on the **atari_spaceinvaders** environment. + +This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a +high throughput on-policy RL framework. I have been using +Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/ + + +## Downloading the model + +After installing Sample-Factory, download the model with: +``` +python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_spaceinvaders +``` + + ## Using the model To run the model after download, use the `enjoy` script corresponding to this environment: diff --git a/checkpoint_p0/best_001932448_494706688_reward_1923.730.pth b/checkpoint_p0/best_001932448_494706688_reward_1923.730.pth new file mode 100644 index 0000000000000000000000000000000000000000..97bbf055adbedf98d854d6fc0d01c9e58efc5ab5 --- /dev/null +++ b/checkpoint_p0/best_001932448_494706688_reward_1923.730.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ceb0464f0adc9927af475225fdb47676393ed242781adf3f4883adf72932cbcd +size 20722035 diff --git a/checkpoint_p0/checkpoint_001965808_505217024.pth b/checkpoint_p0/checkpoint_001965808_505217024.pth new file mode 100644 index 0000000000000000000000000000000000000000..23cb3549ce0860ba78ad88df1856af6cad66b366 --- /dev/null +++ b/checkpoint_p0/checkpoint_001965808_505217024.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a187f974e4cb2a30155bfdc5df193e963a8e0fb08f36c29745e18ad2dc559a7 +size 20722371 diff --git a/checkpoint_p0/checkpoint_001966064_505348096.pth b/checkpoint_p0/checkpoint_001966064_505348096.pth new file mode 100644 index 0000000000000000000000000000000000000000..6034a29711cb604e43df40aa2e4433316450ace5 --- /dev/null +++ b/checkpoint_p0/checkpoint_001966064_505348096.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5d5e0b362861e8911f68561c3cd3b560fad051c5ccea360f3983a3447abc2f1 +size 20722371 diff --git a/checkpoint_p0/milestones/checkpoint_000018272_4677632.pth b/checkpoint_p0/milestones/checkpoint_000018272_4677632.pth new file mode 100644 index 0000000000000000000000000000000000000000..397fe6fd347cfa0d3a79a01dfff179cffefc632c --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000018272_4677632.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bac2d605222a3bd5f42c62158ec1d725633db8f8577a53bcae8a28ea5b1931fa +size 20723163 diff --git a/checkpoint_p0/milestones/checkpoint_000031872_8159232.pth b/checkpoint_p0/milestones/checkpoint_000031872_8159232.pth new file mode 100644 index 0000000000000000000000000000000000000000..a77e3c98fff2cb2e9be0c209d83da7d63f96cfec --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000031872_8159232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a85f8b51aed323db4881c0675ff55a3e2d61a62cb8a822adbf3a6b6df8f7faee +size 20723163 diff --git a/checkpoint_p0/milestones/checkpoint_000045536_11657216.pth b/checkpoint_p0/milestones/checkpoint_000045536_11657216.pth new file mode 100644 index 0000000000000000000000000000000000000000..417515cbe95dea420349a2f64637efc04ecc557b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000045536_11657216.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4410b65ec8022791369baa6d119a98ecc2bf0727351870ac2e0ba9bcdb8ad017 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000059264_15171584.pth b/checkpoint_p0/milestones/checkpoint_000059264_15171584.pth new file mode 100644 index 0000000000000000000000000000000000000000..504b3d4ae1ec940c4b7afe59c9390bda0c971772 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000059264_15171584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b874aa2ab8651073020096358cdb3f5edb0d1c0a553079d4f3715b7c7b4d2df +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000072864_18653184.pth b/checkpoint_p0/milestones/checkpoint_000072864_18653184.pth new file mode 100644 index 0000000000000000000000000000000000000000..97d8a5e5dc9168d4af9866371d5d835de07cae0b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000072864_18653184.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f0078764c77a0f1dc8c1addbcf98d589c02d1ed703729e5711017484a4b19246 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000086528_22151168.pth b/checkpoint_p0/milestones/checkpoint_000086528_22151168.pth new file mode 100644 index 0000000000000000000000000000000000000000..6de8c197a82927890c360e49d39c81a425c4a390 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000086528_22151168.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3cd33244bbdacccab225cb86d1e22036379370705773b3ffe513eaeaa107fffb +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000100256_25665536.pth b/checkpoint_p0/milestones/checkpoint_000100256_25665536.pth new file mode 100644 index 0000000000000000000000000000000000000000..7ea601f077261dd9f5650be2109fff9aefd2cd93 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000100256_25665536.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0deed9782e90708bf02303d3646268546e57b9b44b985bcd940ba3b3f769e949 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000113888_29155328.pth b/checkpoint_p0/milestones/checkpoint_000113888_29155328.pth new file mode 100644 index 0000000000000000000000000000000000000000..f810c9bcc094409cf0ea5baaae077b154f62e106 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000113888_29155328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d8826019028cf1fad211744d9ea350e71df02782e8bd5d587af55a58a5f5a60 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000127520_32645120.pth b/checkpoint_p0/milestones/checkpoint_000127520_32645120.pth new file mode 100644 index 0000000000000000000000000000000000000000..bed8fb63d3766a35e007622b89718822225d5f4b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000127520_32645120.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f6ed99ef2d574e10ab9c5be21dec4f59e87d2905e9492408fdc824eda04c681a +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000141184_36143104.pth b/checkpoint_p0/milestones/checkpoint_000141184_36143104.pth new file mode 100644 index 0000000000000000000000000000000000000000..46c66f1f4821c239381512162715774dcbafc056 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000141184_36143104.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:671338edb608e3f2128e7ea42c38bcf166327bef8c1735cef1b00508aabbcb6b +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000154848_39641088.pth b/checkpoint_p0/milestones/checkpoint_000154848_39641088.pth new file mode 100644 index 0000000000000000000000000000000000000000..ae7a356e314db3ba751834425ad8f2ac2be56d12 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000154848_39641088.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2cb90b1fbf674f79cbfea061e19f0d97ac1c79335b78d6345e55dc3c9408d7ce +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000168512_43139072.pth b/checkpoint_p0/milestones/checkpoint_000168512_43139072.pth new file mode 100644 index 0000000000000000000000000000000000000000..8f9e280b2ac5c3f990afede803d14a1946b2eaba --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000168512_43139072.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:251fe96e7b5f0d02f560c4f065a74c1aada576bc9b94afbb9969887c022da0f1 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000182208_46645248.pth b/checkpoint_p0/milestones/checkpoint_000182208_46645248.pth new file mode 100644 index 0000000000000000000000000000000000000000..83071aa1f097c3c8f0ca9ab487cb3b1e26eebdc6 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000182208_46645248.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:926d5ad211aa58a8112a2ed73df5a57d86f592bf3c40220d629af185009ae3b2 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000195872_50143232.pth b/checkpoint_p0/milestones/checkpoint_000195872_50143232.pth new file mode 100644 index 0000000000000000000000000000000000000000..1ecede3afd7e7c621eb3e01b1f011ff398208630 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000195872_50143232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dcd9bad045d94806d9a46f7841fdd5f011ac142770de16e9d65d9f62cc14e0ea +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000209472_53624832.pth b/checkpoint_p0/milestones/checkpoint_000209472_53624832.pth new file mode 100644 index 0000000000000000000000000000000000000000..9a6edd35a8636710477362887a6ec514c128d2ad --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000209472_53624832.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ff7ed37dc38175cd0aaad7c747ac25f2a3d98d20c69c9b421e40b40945356ba +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000223104_57114624.pth b/checkpoint_p0/milestones/checkpoint_000223104_57114624.pth new file mode 100644 index 0000000000000000000000000000000000000000..60fbd349215af14697c2e4b0066cf1b9a7737535 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000223104_57114624.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:811f5f51fce29462dfdc33165e8844f53d18f6a405de4c9121786115509e63e6 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000236736_60604416.pth b/checkpoint_p0/milestones/checkpoint_000236736_60604416.pth new file mode 100644 index 0000000000000000000000000000000000000000..8c95338b4f8cf4a76ec908e6c4603ba69d106e65 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000236736_60604416.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4f246365facc3f9f3c65ef8f3ea51c383aea466b0d96870140a6b3aaded5e34 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000250368_64094208.pth b/checkpoint_p0/milestones/checkpoint_000250368_64094208.pth new file mode 100644 index 0000000000000000000000000000000000000000..0b5a8da6f7d3de4e318432aa41e2e42fb9da5a7d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000250368_64094208.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea4794091dc79f54a2427b69db27ef67ff24d28531c67e8ab20dd88bde59be06 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000264000_67584000.pth b/checkpoint_p0/milestones/checkpoint_000264000_67584000.pth new file mode 100644 index 0000000000000000000000000000000000000000..b1da54727d0518828c85a3e9a3dd8b55189257cf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000264000_67584000.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8f1be7938fd4d2d7c143dbc8fa0aa69c2b3193b282dcced231575ec7f4fdee66 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000277632_71073792.pth b/checkpoint_p0/milestones/checkpoint_000277632_71073792.pth new file mode 100644 index 0000000000000000000000000000000000000000..a3ffeda4639506a09b207c92febe5a8420619850 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000277632_71073792.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a64fa31a1b454c1c00a192ea47a9cfbdd47eb42bb3ebf9403d9821155fc22aa9 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000291328_74579968.pth b/checkpoint_p0/milestones/checkpoint_000291328_74579968.pth new file mode 100644 index 0000000000000000000000000000000000000000..5f02f8d4e35ea14fdf2b8c8d45c853602d95d4cf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000291328_74579968.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25ec237ca9b0882805f9c3aeaf6c0a290300918a26c26abb4dc35b14592067cc +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000305088_78102528.pth b/checkpoint_p0/milestones/checkpoint_000305088_78102528.pth new file mode 100644 index 0000000000000000000000000000000000000000..e58a2e9a5b64fefdf8a2dcb2e4dea046ccfeb8be --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000305088_78102528.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3a01e27845d1643f1c8186e5e730c7211b0168927b4a4b69b28ba0be5ab4e303 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000318880_81633280.pth b/checkpoint_p0/milestones/checkpoint_000318880_81633280.pth new file mode 100644 index 0000000000000000000000000000000000000000..96cacb36053cbea35a4c89cdd88f408abd4dc785 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000318880_81633280.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f40cd63391e596c49e78bc7a38e586696d063285c276d042bc1c8e09e3a2e56b +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000332640_85155840.pth b/checkpoint_p0/milestones/checkpoint_000332640_85155840.pth new file mode 100644 index 0000000000000000000000000000000000000000..956f9ce00dc71a3a4fa9d4f23a7a0675a2184165 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000332640_85155840.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:49f04ab28ba50e1c1280b9b24bec74e904bb514cfd227bf77bd8ee047c046688 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000346464_88694784.pth b/checkpoint_p0/milestones/checkpoint_000346464_88694784.pth new file mode 100644 index 0000000000000000000000000000000000000000..c2fa66df493f385db5bc740f77e3a7187d5654eb --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000346464_88694784.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8477553ca39ff990c5e2ec0096681336d6067d8a44d847ff9d32900159292cf7 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000360224_92217344.pth b/checkpoint_p0/milestones/checkpoint_000360224_92217344.pth new file mode 100644 index 0000000000000000000000000000000000000000..c5348cf46f8405391fdb3920d4c34f04800bd18f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000360224_92217344.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:433752620842a78c91a68452e593cdb48ba8c96f3f5ea38cf74970e3e3508810 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000374048_95756288.pth b/checkpoint_p0/milestones/checkpoint_000374048_95756288.pth new file mode 100644 index 0000000000000000000000000000000000000000..846d4fd55ef10304601d6253caff017c88e35fe2 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000374048_95756288.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1572567a151396d35fd3a5d966854f24dab34d568b5b172a03ad73dbb62bd323 +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000387872_99295232.pth b/checkpoint_p0/milestones/checkpoint_000387872_99295232.pth new file mode 100644 index 0000000000000000000000000000000000000000..b0732710a0fc4f5142494ff0a6f0ee1d162650af --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000387872_99295232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9e17bb21140f2634d5f8fead23e84770ee07d173a7c129b60600b75c844b7e7e +size 20723219 diff --git a/checkpoint_p0/milestones/checkpoint_000401792_102858752.pth b/checkpoint_p0/milestones/checkpoint_000401792_102858752.pth new file mode 100644 index 0000000000000000000000000000000000000000..2d2e6157c2b1fa973ea667bf925e64e69ca554c3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000401792_102858752.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a34951ba6fe4fc310ccc9e6aa3cdc209a2fddd945fe0d5073ec31d12d15bdbd8 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000415584_106389504.pth b/checkpoint_p0/milestones/checkpoint_000415584_106389504.pth new file mode 100644 index 0000000000000000000000000000000000000000..e438f4e87218fbf08d27dbb3e56fee19d528b939 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000415584_106389504.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c6eedeb3ff72936533eb242407a2f70daf4a6a7c460ff689da50cdfcbe39ff2 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000429408_109928448.pth b/checkpoint_p0/milestones/checkpoint_000429408_109928448.pth new file mode 100644 index 0000000000000000000000000000000000000000..f0c219f998483e166c067b9244fd3de656f48879 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000429408_109928448.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44d45e33130f600bc578bf37e1bf261699077e2a72191c46c5e39bf19b7d3ce0 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000443264_113475584.pth b/checkpoint_p0/milestones/checkpoint_000443264_113475584.pth new file mode 100644 index 0000000000000000000000000000000000000000..1710eb170656700e4e03eb8061f72e536d59178f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000443264_113475584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ecbfc7b7db4dfd010ce88453e870f298c94eea0ad3ca0ab4c70b064416dff03 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000457056_117006336.pth b/checkpoint_p0/milestones/checkpoint_000457056_117006336.pth new file mode 100644 index 0000000000000000000000000000000000000000..dcd235cf909bb14cd779b3171fd7009eeff06827 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000457056_117006336.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4963d3a2a1a31fb7ac8e926acde235b2c219c16d87f01ce82f67f866fc4a76d +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000470912_120553472.pth b/checkpoint_p0/milestones/checkpoint_000470912_120553472.pth new file mode 100644 index 0000000000000000000000000000000000000000..596cadf11b7f08f365d8c871e3ff6c400a4cf765 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000470912_120553472.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ef913a2e4db5fc04e1fd9e441c19dc42e72bb9bdb2c98c860fd7012776c36909 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000484704_124084224.pth b/checkpoint_p0/milestones/checkpoint_000484704_124084224.pth new file mode 100644 index 0000000000000000000000000000000000000000..488354be5045085eab107892b272f3d23bc96bfe --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000484704_124084224.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c8efdb2f180232582fb5cb3232d6e70fa1bd0acd93be82732863c754b5db2f8 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000498560_127631360.pth b/checkpoint_p0/milestones/checkpoint_000498560_127631360.pth new file mode 100644 index 0000000000000000000000000000000000000000..e5daad50f8d3d4616083dc9a9c50b6edcaecc4f8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000498560_127631360.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f180d27aacaf5cd6cab145382f1d1d2163cb40c3cb2114edcd37acc8fa18aaf +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000512384_131170304.pth b/checkpoint_p0/milestones/checkpoint_000512384_131170304.pth new file mode 100644 index 0000000000000000000000000000000000000000..1873c452164b833c63379f19e50023931e13a446 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000512384_131170304.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4afa8b94450d39e1b3d4bc454c9f490ead8c6900aece5abf00ccef6d55165eac +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000526208_134709248.pth b/checkpoint_p0/milestones/checkpoint_000526208_134709248.pth new file mode 100644 index 0000000000000000000000000000000000000000..12793b240568513433116e99e00802b323993ed2 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000526208_134709248.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:93a66fd8b39487eca4bd7e2ac932a5b8b7ce2e01a0fdfb540d0d89ebb76eee79 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000540064_138256384.pth b/checkpoint_p0/milestones/checkpoint_000540064_138256384.pth new file mode 100644 index 0000000000000000000000000000000000000000..8fb3801eab860eb0984073ad896217a819c34e6f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000540064_138256384.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f05217e5ecb746aa20aafe7fd5f007cd664c6ce00fa390a9f07326906b2aed4 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000553888_141795328.pth b/checkpoint_p0/milestones/checkpoint_000553888_141795328.pth new file mode 100644 index 0000000000000000000000000000000000000000..6729e295ac733c3d9878809222bdc7ef487f0d3a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000553888_141795328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fb2574ce87ed6dfc8d6a6265a076dbafb4ec2e0b1ffca550033a398ada6b3b9 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000567712_145334272.pth b/checkpoint_p0/milestones/checkpoint_000567712_145334272.pth new file mode 100644 index 0000000000000000000000000000000000000000..1c564c621a34ba66a48e578c137c8b3cc25afdad --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000567712_145334272.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:49afc810faee3e7a530f9664f1d77269d710ab3e121f007ecbb0242646abcb95 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000581568_148881408.pth b/checkpoint_p0/milestones/checkpoint_000581568_148881408.pth new file mode 100644 index 0000000000000000000000000000000000000000..58c66e27f70a225e72cae33e64cb7aeff40e661f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000581568_148881408.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:17a0f3f41724082a6f9904e0fa1316e9b7f5e7b285fa62c104e9aefe681dbd60 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000595392_152420352.pth b/checkpoint_p0/milestones/checkpoint_000595392_152420352.pth new file mode 100644 index 0000000000000000000000000000000000000000..2265300a29ab9d19e55461799ac10e0e368aee4a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000595392_152420352.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:796e4c74e6b9c033531736417c9d95f5160cf8f58c024441885b902f6e9b78fa +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000609248_155967488.pth b/checkpoint_p0/milestones/checkpoint_000609248_155967488.pth new file mode 100644 index 0000000000000000000000000000000000000000..3176ac88cfca9a3fc416c09b988ba7342d121483 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000609248_155967488.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ed72cb7d5e5de9e170e9c90bed50112293cf53dfa04baab32b8db338acd6eb5 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000623072_159506432.pth b/checkpoint_p0/milestones/checkpoint_000623072_159506432.pth new file mode 100644 index 0000000000000000000000000000000000000000..d3eaeebb2fd60b4cddd28b3b8cddf98b07af1050 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000623072_159506432.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:20856205803d55f2683cd3390ef7403d596eef802884047436d3d4f0d16628d9 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000636864_163037184.pth b/checkpoint_p0/milestones/checkpoint_000636864_163037184.pth new file mode 100644 index 0000000000000000000000000000000000000000..3cbba9039074034c72030cee187024e580f7ec7d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000636864_163037184.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:179bcae341c31b426883053b527483febcb010de59cdc53124ad3ccf43243a0b +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000650688_166576128.pth b/checkpoint_p0/milestones/checkpoint_000650688_166576128.pth new file mode 100644 index 0000000000000000000000000000000000000000..b06c371109de216fc6a28fa07c1912d76d04e7a4 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000650688_166576128.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:568249712df545de9e943c15f408ae507994ff2920dec7778c1152989837119c +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000664384_170082304.pth b/checkpoint_p0/milestones/checkpoint_000664384_170082304.pth new file mode 100644 index 0000000000000000000000000000000000000000..51046b1a9856cbff587c15e287c615fb77589f37 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000664384_170082304.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:17c4176bdecfee12f75892b9ddcf544f8082ffcb270cb8f91bc9885c43e21be6 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000678144_173604864.pth b/checkpoint_p0/milestones/checkpoint_000678144_173604864.pth new file mode 100644 index 0000000000000000000000000000000000000000..a78895821e914669a4d37ab822bb56925c17f876 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000678144_173604864.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f49d1805f9f06d7a58090b461ae95cc3f1ee9db06a8cab05c04373ded19c20d2 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000691840_177111040.pth b/checkpoint_p0/milestones/checkpoint_000691840_177111040.pth new file mode 100644 index 0000000000000000000000000000000000000000..96965e76dfe0ef4d170a4c3627bb17778d727ee5 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000691840_177111040.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c3a7928c839f677c42f0379c93d91e8e8d7d3063efaf8e46793f9e22ce96d55 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000705120_180510720.pth b/checkpoint_p0/milestones/checkpoint_000705120_180510720.pth new file mode 100644 index 0000000000000000000000000000000000000000..33d0dd571fd2c4b0dd18d766961377c4bc673f47 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000705120_180510720.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9fad0ec0b12d1e3ab34bbe93aff8e6c04ac3fe2c9593492496b3bf4ad26a600 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000718624_183967744.pth b/checkpoint_p0/milestones/checkpoint_000718624_183967744.pth new file mode 100644 index 0000000000000000000000000000000000000000..9cd6f08d652efd0ce0c2665cf8996c97d84e5f4d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000718624_183967744.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1efc84a3ef43235bd3a2453d3ac70e3065f2971ddd43f9d7b7481c985566fb0c +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000732320_187473920.pth b/checkpoint_p0/milestones/checkpoint_000732320_187473920.pth new file mode 100644 index 0000000000000000000000000000000000000000..c85c75e04b599283238f27ee8ecd460c7b0643d5 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000732320_187473920.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2adce4e3b98d07165d21bd7225b89b3b4c5c011af623ac825ddef16b2f53d280 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000745952_190963712.pth b/checkpoint_p0/milestones/checkpoint_000745952_190963712.pth new file mode 100644 index 0000000000000000000000000000000000000000..3857ed1f36e08f9395bd5de7ad85880b74cd139a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000745952_190963712.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a0d600cb8ddbf6b795c5f894d17b2675250101c761c0977a9366299d4867085 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000759680_194478080.pth b/checkpoint_p0/milestones/checkpoint_000759680_194478080.pth new file mode 100644 index 0000000000000000000000000000000000000000..7b80fdf38b6c7bae99e9a995f34834b13850cfe6 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000759680_194478080.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f75eaa31a1f2c11ce8ec77d15e0c615fd25abb753c638953c9d3d6c49c7a5041 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000773440_198000640.pth b/checkpoint_p0/milestones/checkpoint_000773440_198000640.pth new file mode 100644 index 0000000000000000000000000000000000000000..14ce0b24a2c897a3b611fdb24cda67b466b27e9a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000773440_198000640.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e0a298281b43dafc959e88e50877d8cb8d9bb0cee1275419443933521141a966 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000787264_201539584.pth b/checkpoint_p0/milestones/checkpoint_000787264_201539584.pth new file mode 100644 index 0000000000000000000000000000000000000000..b5391d9875ac0e2ae6e09b3a77fb3f67d4df2bc1 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000787264_201539584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:479b10ee889b0c8ccc8dcf725fdb45c98a32e88692723d0b5a3332181365f7ae +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000801152_205094912.pth b/checkpoint_p0/milestones/checkpoint_000801152_205094912.pth new file mode 100644 index 0000000000000000000000000000000000000000..7d7d60392748b2a467f0fb61aa28ab1f76a7b494 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000801152_205094912.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:87d7464aca1fce2058838dced5ccf9137598fa31834a1e1f711a41e0c68a8630 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000814976_208633856.pth b/checkpoint_p0/milestones/checkpoint_000814976_208633856.pth new file mode 100644 index 0000000000000000000000000000000000000000..e025a1a9d2d121b2f178d5f450407360a415fbbf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000814976_208633856.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:996eaf6e1d3e9833fe3cea4b9d9284461c65d7d8d35e261351eb2bd7bf44e81e +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000828864_212189184.pth b/checkpoint_p0/milestones/checkpoint_000828864_212189184.pth new file mode 100644 index 0000000000000000000000000000000000000000..d97078251a709e926f4d4f3f7c2f52abd7288b5b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000828864_212189184.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:420565f36d5015f5b9e35b8139a53b32f578ee2ac2dc41cb096658c3fc059f28 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000842720_215736320.pth b/checkpoint_p0/milestones/checkpoint_000842720_215736320.pth new file mode 100644 index 0000000000000000000000000000000000000000..5db756906c9d3026351b48ff70e483477d874aa3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000842720_215736320.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:03e6abed62417ac503ac4fe9d52602d88f696f52cced9e37f5372d6414d6a75c +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000856608_219291648.pth b/checkpoint_p0/milestones/checkpoint_000856608_219291648.pth new file mode 100644 index 0000000000000000000000000000000000000000..90f5280f1eba942160cf5197a55b6c838527223a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000856608_219291648.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1cd893e59d77cd2e0ccf5d1bc2683acb4c8829bf77153320f08dc2496fac4096 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000870432_222830592.pth b/checkpoint_p0/milestones/checkpoint_000870432_222830592.pth new file mode 100644 index 0000000000000000000000000000000000000000..b78da7461d2b42071d1851de68cbbfbc2e92a64f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000870432_222830592.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3ea9093e928175ba19da5c4f4179acacb591719de3d4669e1b0f0e35fdd8f99 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000884320_226385920.pth b/checkpoint_p0/milestones/checkpoint_000884320_226385920.pth new file mode 100644 index 0000000000000000000000000000000000000000..7d073b0316b508b3d6811238e9cfc7c483a747e5 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000884320_226385920.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:015cb56bf17dae66f217081c6d05e3858ba047b3561f349f06a8ba4ecd618df3 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000898144_229924864.pth b/checkpoint_p0/milestones/checkpoint_000898144_229924864.pth new file mode 100644 index 0000000000000000000000000000000000000000..f8907e5b7970497fc91269883db3470687363638 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000898144_229924864.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:424dd6a5d37648ad5a9419d0e380716e0d284ff346721282be3a65e274128674 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000912032_233480192.pth b/checkpoint_p0/milestones/checkpoint_000912032_233480192.pth new file mode 100644 index 0000000000000000000000000000000000000000..d09dcd0fde2da852811c26f9bcd60a8705ef8eb6 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000912032_233480192.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c35bfe779bf8061c3246b0acc6bb643fa4edce25b60f169de512f1158140d6eb +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000925888_237027328.pth b/checkpoint_p0/milestones/checkpoint_000925888_237027328.pth new file mode 100644 index 0000000000000000000000000000000000000000..75177312cee1630abdb1183d5c92bbaa6ab9c3c7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000925888_237027328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc156dc41537df472e22872abf87c16bf7563917c23c7e932be513a66c847f1b +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000939808_240590848.pth b/checkpoint_p0/milestones/checkpoint_000939808_240590848.pth new file mode 100644 index 0000000000000000000000000000000000000000..b1ee770a2f969f917301d7a393ef937ef5855316 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000939808_240590848.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0aebde09c372e02f06241de76da3d6b3c4410857cc1736ae503c7c3d2f3115b8 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000953664_244137984.pth b/checkpoint_p0/milestones/checkpoint_000953664_244137984.pth new file mode 100644 index 0000000000000000000000000000000000000000..b87c0d4aca7bc1430d5912b757e334838bbf812a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000953664_244137984.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:885156c329b31b991e17715f4217006058ce0ffa1a27fb682f52bc7999c0a064 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000967488_247676928.pth b/checkpoint_p0/milestones/checkpoint_000967488_247676928.pth new file mode 100644 index 0000000000000000000000000000000000000000..966e43a29731d53e5e6bb7efa6fcd47f92078e00 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000967488_247676928.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd0b8119155b4c97b5c0fe760a1198e43d2010b532dde4b4d263e609225069b8 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000981312_251215872.pth b/checkpoint_p0/milestones/checkpoint_000981312_251215872.pth new file mode 100644 index 0000000000000000000000000000000000000000..9d9cd07c0ad9ad54c2bb73f8336f4fb36b193184 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000981312_251215872.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a2ccaea7aa5da28c445c60d5330208016c3d0e8fc7487c50340ebfedc2580a48 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_000995168_254763008.pth b/checkpoint_p0/milestones/checkpoint_000995168_254763008.pth new file mode 100644 index 0000000000000000000000000000000000000000..10bd4cf59c72e7a6e1e2e59a8677f7c503354848 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000995168_254763008.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5bec889b24a274aaf4dec1c31858738076b1cdde75e23c5b72775a2c5504b895 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001009024_258310144.pth b/checkpoint_p0/milestones/checkpoint_001009024_258310144.pth new file mode 100644 index 0000000000000000000000000000000000000000..20d3ed7593e783ce1b96691b5eb7b1f639ffcef8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001009024_258310144.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65608d7649c289a0c232808c5c7b9a2a85c3f9461c2c17deda1dcc2c692428ca +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001022816_261840896.pth b/checkpoint_p0/milestones/checkpoint_001022816_261840896.pth new file mode 100644 index 0000000000000000000000000000000000000000..6ceb81229d3c9fd09a0dd0b9298ab64f648c4b78 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001022816_261840896.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00628a3afdabc0486cb7928880edae80aef95a15ea03563440ee3439385d8341 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001036640_265379840.pth b/checkpoint_p0/milestones/checkpoint_001036640_265379840.pth new file mode 100644 index 0000000000000000000000000000000000000000..0df201e15a2df5af464880c49e61c9fbb2b40f65 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001036640_265379840.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2c217d7b732611c957a05f0dd65ad4a6cbafcb9843fa280c22281c50ad2b52b5 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001050528_268935168.pth b/checkpoint_p0/milestones/checkpoint_001050528_268935168.pth new file mode 100644 index 0000000000000000000000000000000000000000..5a9efe3983e5e548232bb02e564fca0055267f0e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001050528_268935168.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1d0982e565e484bcd83614902c303f9b2d40022b025652fbe7aa1c76ccea55f +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001064384_272482304.pth b/checkpoint_p0/milestones/checkpoint_001064384_272482304.pth new file mode 100644 index 0000000000000000000000000000000000000000..e9256e4e2ea594d362e2767e978e332d5d451ee8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001064384_272482304.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fbdeadb90ad19f150f28ced0f1c4d5bb4759aeea803ecaba41be106421318064 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001078176_276013056.pth b/checkpoint_p0/milestones/checkpoint_001078176_276013056.pth new file mode 100644 index 0000000000000000000000000000000000000000..2afc7b677646c0e214cb6d0e277288b059e898a1 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001078176_276013056.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f9281abd612d49429f73218dd3413247e1fc734e8b3244c598bd2e1c27cc16cc +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001091712_279478272.pth b/checkpoint_p0/milestones/checkpoint_001091712_279478272.pth new file mode 100644 index 0000000000000000000000000000000000000000..9c484c6a93296ad34a09e9e30932fb9a66fb37fc --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001091712_279478272.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a9cfcc2d2b78975373e1aa5ef41c42aa557c094acc62a2a1aa29405e161373b +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001105472_283000832.pth b/checkpoint_p0/milestones/checkpoint_001105472_283000832.pth new file mode 100644 index 0000000000000000000000000000000000000000..1787d108f97844871659ab75a87254c9f32608f1 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001105472_283000832.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1da47bf6d5b0663440de1f50cc395f8f3a510d37afe68e9f8124852b90a82bed +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001119232_286523392.pth b/checkpoint_p0/milestones/checkpoint_001119232_286523392.pth new file mode 100644 index 0000000000000000000000000000000000000000..e121bd8f1af799abd450ae2f29d1bf999279de52 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001119232_286523392.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:05e557a8997715aad6eb11ccb37a58bdd350c1a10c47d6a714649819c3baf292 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001132992_290045952.pth b/checkpoint_p0/milestones/checkpoint_001132992_290045952.pth new file mode 100644 index 0000000000000000000000000000000000000000..2a0d98fffe857c501852bcfe0bd754cb689c8d10 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001132992_290045952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:05f3fc065531148806fe98927c41de538284f8632399c93abf034a2bc5f6e6d4 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001146784_293576704.pth b/checkpoint_p0/milestones/checkpoint_001146784_293576704.pth new file mode 100644 index 0000000000000000000000000000000000000000..149a0646d0caa0c855bce7e5a3bef4e246f3cec3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001146784_293576704.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:67425bd8dc5e9dfc5f761b85b1f7d9273c0818d5fb38f45424c4271c6a8c9b5e +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001160512_297091072.pth b/checkpoint_p0/milestones/checkpoint_001160512_297091072.pth new file mode 100644 index 0000000000000000000000000000000000000000..11c3a69809f0965a57b91596fd482fc38f9d11ce --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001160512_297091072.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41429ac41657f263bd5498bc4dfe06c2b4b6ceae82706e7de64c14a8ba8284c4 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001173984_300539904.pth b/checkpoint_p0/milestones/checkpoint_001173984_300539904.pth new file mode 100644 index 0000000000000000000000000000000000000000..2c536558ef7d39dbf422e28b7017f95f1cab6465 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001173984_300539904.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5b5898c3ab4eee54d671ad4c3a96c6909784c0ce9ae40b836da16998cf55f1b9 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001187648_304037888.pth b/checkpoint_p0/milestones/checkpoint_001187648_304037888.pth new file mode 100644 index 0000000000000000000000000000000000000000..a881208e4e294f07e221f8286a0ad7f7056b89b7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001187648_304037888.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d41fe992424382b6fa76f11f6042b73f736ee5d77b024d9279656d97729eb722 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001201440_307568640.pth b/checkpoint_p0/milestones/checkpoint_001201440_307568640.pth new file mode 100644 index 0000000000000000000000000000000000000000..038dc99064c2587912cd8ed68ce399f806b5704e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001201440_307568640.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d3dd5229506fccfca475da4f5a78b96d1c723fcbdce8def41c00fe7a5d1ec63 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001215136_311074816.pth b/checkpoint_p0/milestones/checkpoint_001215136_311074816.pth new file mode 100644 index 0000000000000000000000000000000000000000..f48ff2c20b895ce23d07a4a59693d5985d3f1dfc --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001215136_311074816.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:259c220e327a6d52226b08ad097fe1e0d18aff2382ee71113b663a0cfc4dfa0c +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001228800_314572800.pth b/checkpoint_p0/milestones/checkpoint_001228800_314572800.pth new file mode 100644 index 0000000000000000000000000000000000000000..89b6bf80b4e01ec5eb59de392bd0476b7ad98886 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001228800_314572800.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae83c5a5a6471e29352ee22aaebead3ef6e748a722ce8207ddc9d4bdd1d1b731 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001242560_318095360.pth b/checkpoint_p0/milestones/checkpoint_001242560_318095360.pth new file mode 100644 index 0000000000000000000000000000000000000000..1b1c04ffc279a7ac4e2781ba237c2b87437e3bf3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001242560_318095360.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:020d39e4f63b56f95ddd2521083866d9654b043d4afe4bd5ba91cbacd3422da4 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001256224_321593344.pth b/checkpoint_p0/milestones/checkpoint_001256224_321593344.pth new file mode 100644 index 0000000000000000000000000000000000000000..807fee6001372f76ece407126444df9bf564b68e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001256224_321593344.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc11ac3005127645e449cd45f1cce6fcee71ab19eabaafb543994bc9c2222155 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001269888_325091328.pth b/checkpoint_p0/milestones/checkpoint_001269888_325091328.pth new file mode 100644 index 0000000000000000000000000000000000000000..1de14201e6b7ca0f2c183c1f2e193cb6f043709d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001269888_325091328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53645b4ab39ffc22b085e001ff7d1e4a8ce0cc23603bcc9d72e8c476b3b254a1 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001283584_328597504.pth b/checkpoint_p0/milestones/checkpoint_001283584_328597504.pth new file mode 100644 index 0000000000000000000000000000000000000000..a86e2e97d037c231ce8107e2114a787aad447523 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001283584_328597504.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b7d4e3d017eb041286c2b08766e8a1a32970cee5666970f3d245d9b61d9fe5c +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001297248_332095488.pth b/checkpoint_p0/milestones/checkpoint_001297248_332095488.pth new file mode 100644 index 0000000000000000000000000000000000000000..e96899dc452978bea8e5e731f69dad5a8f68512d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001297248_332095488.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09a0878c41b0af37fd04c5810c86b66c54387bd2ef75f6439ae2c2aee94b9cfa +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001310912_335593472.pth b/checkpoint_p0/milestones/checkpoint_001310912_335593472.pth new file mode 100644 index 0000000000000000000000000000000000000000..6d51dadf958d3d1422562091672861889f342d86 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001310912_335593472.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a0563860d3809ca48174b8397251b96a573175ae15e817beea86b426f2a1eef +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001324576_339091456.pth b/checkpoint_p0/milestones/checkpoint_001324576_339091456.pth new file mode 100644 index 0000000000000000000000000000000000000000..639b289ac784a8f0b02e7d407a10f5485829063c --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001324576_339091456.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d60634edbc2c61dd7f6a14b622484b5a5aa75996e2b586cb8282874c39fe3b15 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001338240_342589440.pth b/checkpoint_p0/milestones/checkpoint_001338240_342589440.pth new file mode 100644 index 0000000000000000000000000000000000000000..8fc55470398f3b607239c8251629e25616f07256 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001338240_342589440.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2110f25540e670cc94d64dbcbc38cc183b98368dcad9764f470cc43e804c2fc +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001351936_346095616.pth b/checkpoint_p0/milestones/checkpoint_001351936_346095616.pth new file mode 100644 index 0000000000000000000000000000000000000000..336e4d074a9cb2dccd0b337c7267137649e59657 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001351936_346095616.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:317ad59c349a1190436eeb0caeea1f24641470852d40422655f37a93f44b7c86 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001365664_349609984.pth b/checkpoint_p0/milestones/checkpoint_001365664_349609984.pth new file mode 100644 index 0000000000000000000000000000000000000000..b4f50e6686b558759d1015ad4462f6f3fb8f15cb --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001365664_349609984.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28e5554d2164eee05ecc660f64dfadd29d6cc18e1a458ca7b98e65dd593fe2be +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001379456_353140736.pth b/checkpoint_p0/milestones/checkpoint_001379456_353140736.pth new file mode 100644 index 0000000000000000000000000000000000000000..e537a4dbb1d70a2c32c11684011b53f8324524aa --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001379456_353140736.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4e5370a128193d4f9b7af78d2b27887584d109a2f3b74583cf19772778abdebb +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001393216_356663296.pth b/checkpoint_p0/milestones/checkpoint_001393216_356663296.pth new file mode 100644 index 0000000000000000000000000000000000000000..908126c0d9d8dbcc4b9716e7e540a9be4d2fc648 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001393216_356663296.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b761c2cb66de7827b32fec4ed63ed31343fa0fc8b24e079b8b50757979578fbc +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001406976_360185856.pth b/checkpoint_p0/milestones/checkpoint_001406976_360185856.pth new file mode 100644 index 0000000000000000000000000000000000000000..f59ec715719a8645d05ffc96719cc4ea7ad280c8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001406976_360185856.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cdf0c6a24be828d0d32fb7bddcca2f4a176014779059d8276139192e2d6f23a9 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001420736_363708416.pth b/checkpoint_p0/milestones/checkpoint_001420736_363708416.pth new file mode 100644 index 0000000000000000000000000000000000000000..2ec131def655e759e1dc0f88bf034d70e158b5e3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001420736_363708416.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4392103356637e9dcd2c5c81ad599ab802c469fcd815161863a6966f1c311e4e +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001434496_367230976.pth b/checkpoint_p0/milestones/checkpoint_001434496_367230976.pth new file mode 100644 index 0000000000000000000000000000000000000000..cc17194332dbfa52bc950e3ecd938627fea96c0b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001434496_367230976.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c76825e20d74914ee5539fc1e1ec203e92ca4acdee750e7bd8de453b6f66668 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001448256_370753536.pth b/checkpoint_p0/milestones/checkpoint_001448256_370753536.pth new file mode 100644 index 0000000000000000000000000000000000000000..1424eef0c2eee63601b621ada5fe5c99c92b3f6d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001448256_370753536.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3036dda247afa429d010c3f5f70b1a950d43791d81204af433e61adbe260a6b +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001462016_374276096.pth b/checkpoint_p0/milestones/checkpoint_001462016_374276096.pth new file mode 100644 index 0000000000000000000000000000000000000000..0547307d250fd5565d90b5f62f201ba3efda30f6 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001462016_374276096.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a965bf0e80dc6aa5c7d2a9809ba08a2b8c4316192c2d749e78c0a660e694fc03 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001475744_377790464.pth b/checkpoint_p0/milestones/checkpoint_001475744_377790464.pth new file mode 100644 index 0000000000000000000000000000000000000000..c278e7be23da7fba3c7ea74d48b86ff0383368ca --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001475744_377790464.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41c7b182a55908f74394dd6226561f747f15461dcdda4ba83b2be3f760ac11e8 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001489536_381321216.pth b/checkpoint_p0/milestones/checkpoint_001489536_381321216.pth new file mode 100644 index 0000000000000000000000000000000000000000..0255f74291acc387616dc7580cf30223408c2de8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001489536_381321216.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:864584602891487d36798adfbad5627f4f36a563de7a7cd1c164af506cd420d7 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001503328_384851968.pth b/checkpoint_p0/milestones/checkpoint_001503328_384851968.pth new file mode 100644 index 0000000000000000000000000000000000000000..88f030b3be853c04322b90af1e57cdc57ec8f3bf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001503328_384851968.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2c8b6cd18ff90a7d30ac27f725168555635f8679b33aefdb7768e39cdd13a37 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001517088_388374528.pth b/checkpoint_p0/milestones/checkpoint_001517088_388374528.pth new file mode 100644 index 0000000000000000000000000000000000000000..9c45e936f356c391d402c27e0e891c6b1821ce46 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001517088_388374528.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fe0927a2ec4d521d484ccb10b9fde8be9356f69b045c670f2def9983e853f848 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001530976_391929856.pth b/checkpoint_p0/milestones/checkpoint_001530976_391929856.pth new file mode 100644 index 0000000000000000000000000000000000000000..be1dba4fbe7d75833d1a61544233db833a679a05 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001530976_391929856.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:12ecd1af99b3b5491b266f925bf4d11ff4ad082943ceaab86ae60d8529727869 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001544864_395485184.pth b/checkpoint_p0/milestones/checkpoint_001544864_395485184.pth new file mode 100644 index 0000000000000000000000000000000000000000..ff881e5824f430a50e7cac10750ddea152c92d9e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001544864_395485184.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9b4ffce86ebfec807d3facf20c399937e81659daeab501212625c887901baa83 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001558784_399048704.pth b/checkpoint_p0/milestones/checkpoint_001558784_399048704.pth new file mode 100644 index 0000000000000000000000000000000000000000..c1d2a18e75d544abaaa7eed0f0f309daf321e9fa --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001558784_399048704.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:67f72065facde17dfeb1512edecbf958cff4169e8b857c5398cb550c9a3faa58 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001572672_402604032.pth b/checkpoint_p0/milestones/checkpoint_001572672_402604032.pth new file mode 100644 index 0000000000000000000000000000000000000000..1d6b73b80f0c3121ed20e5bdd5a515587affef2e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001572672_402604032.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7982fb07d4d97427c1cc38ab804e8dbdab88d2afc01d44da74709d5aa5f383e2 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001586528_406151168.pth b/checkpoint_p0/milestones/checkpoint_001586528_406151168.pth new file mode 100644 index 0000000000000000000000000000000000000000..8acb69bd9e51780c23b8f7ea7652843502221953 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001586528_406151168.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0cfdf78e90ec74ccee781566589b5294434adb6764c80cab0df35958a859f1f2 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001600384_409698304.pth b/checkpoint_p0/milestones/checkpoint_001600384_409698304.pth new file mode 100644 index 0000000000000000000000000000000000000000..cb313242f46f63358a4af8ca780ebf01ab940b2e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001600384_409698304.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc186f99a640106729cdebcc6096db4a502993cc0f91affebf4ca05471973d13 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001614208_413237248.pth b/checkpoint_p0/milestones/checkpoint_001614208_413237248.pth new file mode 100644 index 0000000000000000000000000000000000000000..4b8583436f51513c9c1dff2fa7888433c3859f98 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001614208_413237248.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:956481e131c2692ef437a4fcc5d0812f0591492bd060082db5d8e49382e0a32b +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001628032_416776192.pth b/checkpoint_p0/milestones/checkpoint_001628032_416776192.pth new file mode 100644 index 0000000000000000000000000000000000000000..62828f62d90f093c4a76b9399f902a731559a0a9 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001628032_416776192.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:06b525c73a8cee4e44be9cee75272423f114359ddf16a5b2aa9e0a425423a5fa +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001641856_420315136.pth b/checkpoint_p0/milestones/checkpoint_001641856_420315136.pth new file mode 100644 index 0000000000000000000000000000000000000000..b41953bec3ecc1271ce0de59aebd96fb97468901 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001641856_420315136.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b32cf6ffb36b602e5dc878f9c40e337e921f863fea020f7bb811e85d6fb730ef +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001655744_423870464.pth b/checkpoint_p0/milestones/checkpoint_001655744_423870464.pth new file mode 100644 index 0000000000000000000000000000000000000000..b446414e31da180481a635099e7bc8c9d4d259b4 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001655744_423870464.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d058f3c4de3f7e544016873fce005591def2b342ae51193ae624aeebd49b1bf8 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001669632_427425792.pth b/checkpoint_p0/milestones/checkpoint_001669632_427425792.pth new file mode 100644 index 0000000000000000000000000000000000000000..601ac424b96975f2b9dea378416a37d9ac5604b3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001669632_427425792.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e81a82b0ebee0912b7472d6da9dc944912b2a596e1eaf3a5adc524dd32d19d1f +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001683520_430981120.pth b/checkpoint_p0/milestones/checkpoint_001683520_430981120.pth new file mode 100644 index 0000000000000000000000000000000000000000..f11791f87932647a8873477d6d23864cf12b3d99 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001683520_430981120.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c7e6e308ea6644f1e5461228497d9d3b9ea87ad4e33d93cc8fb97d218a78b440 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001696992_434429952.pth b/checkpoint_p0/milestones/checkpoint_001696992_434429952.pth new file mode 100644 index 0000000000000000000000000000000000000000..9cfe52cefb42dd6cce383dcaa9e6cab1171b77fc --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001696992_434429952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:01dd7d59b7750ad4395f2649b01175c79f92f10ccbb7c7aa28297c4727601f74 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001710624_437919744.pth b/checkpoint_p0/milestones/checkpoint_001710624_437919744.pth new file mode 100644 index 0000000000000000000000000000000000000000..3bb17fe5006748f8a66b9e95471d633a5c709614 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001710624_437919744.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d21b0a78310322db4341af11698f19e2b68fb5eb27b10320e1121bbc9355f949 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001724448_441458688.pth b/checkpoint_p0/milestones/checkpoint_001724448_441458688.pth new file mode 100644 index 0000000000000000000000000000000000000000..8314fdf8e794bac3636752dc0f610fbc21550072 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001724448_441458688.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b55f92aa78b9502c285507a6baf6de9c86e5f981fbd714ce2075b4b24ab5a38 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001738304_445005824.pth b/checkpoint_p0/milestones/checkpoint_001738304_445005824.pth new file mode 100644 index 0000000000000000000000000000000000000000..b5ad78d2fa9618d2cbc998fd8de9bb0ddaeee588 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001738304_445005824.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32c5d45c9d5ba4c753fbb20e8c9469d851e6c678833abd44de62bdd2affa0275 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001752160_448552960.pth b/checkpoint_p0/milestones/checkpoint_001752160_448552960.pth new file mode 100644 index 0000000000000000000000000000000000000000..7e1005bdd7eeb2733ccb707cb5f2bc75e8268234 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001752160_448552960.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:46e6fc3f18b9ae1baaa403ceba64e7d47c26766e2bb4190fdd41367c5c7ec470 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001766080_452116480.pth b/checkpoint_p0/milestones/checkpoint_001766080_452116480.pth new file mode 100644 index 0000000000000000000000000000000000000000..d8a79e0f0c7f36e6d5088c3d19f93cf10e920fa4 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001766080_452116480.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1af73c9de2e8ae3723376764394c1e169dcb5fa60b68d9d721ed6e98a836d4e4 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001779936_455663616.pth b/checkpoint_p0/milestones/checkpoint_001779936_455663616.pth new file mode 100644 index 0000000000000000000000000000000000000000..51b6636cea9f73dea5c4ffadb61c38df1fb06a07 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001779936_455663616.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1e534a090537cc54e9abe300038b145960d96a7539c454d5a296cc81560fc6b +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001793824_459218944.pth b/checkpoint_p0/milestones/checkpoint_001793824_459218944.pth new file mode 100644 index 0000000000000000000000000000000000000000..d3c1a235cd764ba69dbf79841e3d0cf92504c974 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001793824_459218944.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aec95273124a782626e9b7b025df6a8378de625bbc57b5acf7458aa0def76a2c +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001807712_462774272.pth b/checkpoint_p0/milestones/checkpoint_001807712_462774272.pth new file mode 100644 index 0000000000000000000000000000000000000000..057e59d2dbcf097eda493763e63025e10522890c --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001807712_462774272.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e4da3806b5cbc6156c316a201bae0f154e8d4ea62a2163a06d5dc0bc885aecd5 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001821600_466329600.pth b/checkpoint_p0/milestones/checkpoint_001821600_466329600.pth new file mode 100644 index 0000000000000000000000000000000000000000..9f8e432e446db176518a4660985663813834693e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001821600_466329600.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5893aa414f33ba996ac5d1c29ff8ad2756257461d50d83bb18d4625b65d91f53 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001835424_469868544.pth b/checkpoint_p0/milestones/checkpoint_001835424_469868544.pth new file mode 100644 index 0000000000000000000000000000000000000000..3a864ec47ff7ad6ed85319ea0d062b33cfb5b133 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001835424_469868544.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32a8e17b5bd266233a0fa9ea8411fa22c5492edc3181caab7a0306c6c5f1fd9a +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001849280_473415680.pth b/checkpoint_p0/milestones/checkpoint_001849280_473415680.pth new file mode 100644 index 0000000000000000000000000000000000000000..22fd21e29c725c3f9dab73339c5d703e8b3635a2 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001849280_473415680.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f76ad7b981d7baccab770e4757273fc8305be52a45b796586eb94dde8492cb2 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001863040_476938240.pth b/checkpoint_p0/milestones/checkpoint_001863040_476938240.pth new file mode 100644 index 0000000000000000000000000000000000000000..46e411a75d01a1711087640ebf7f34f03839c332 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001863040_476938240.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:803baa10eb4da98d276e120abc6df2ea8ab70a77798255125a4cff3e77c02e33 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001876896_480485376.pth b/checkpoint_p0/milestones/checkpoint_001876896_480485376.pth new file mode 100644 index 0000000000000000000000000000000000000000..d47fff81377f13f4d8efbef521aad67116b982c7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001876896_480485376.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cf7e19601a633037b13da70a65280f1e0943539a1f6e8095c3289671f8041f3d +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001890752_484032512.pth b/checkpoint_p0/milestones/checkpoint_001890752_484032512.pth new file mode 100644 index 0000000000000000000000000000000000000000..0e3badc0bc0b7471a0b7c1768dcef453d90b2edf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001890752_484032512.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5bf6d4cf4845cd1eeea7d957ca3ccbfd2d930b8078216c374184d68849e08a56 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001904672_487596032.pth b/checkpoint_p0/milestones/checkpoint_001904672_487596032.pth new file mode 100644 index 0000000000000000000000000000000000000000..c57bf3ba1923f3c1a7a57b6ad6f411b90a986812 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001904672_487596032.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:042ff5cff209387ab4d2a11060cee5df4e40f73c7e549aaa7664b5199567927c +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001918560_491151360.pth b/checkpoint_p0/milestones/checkpoint_001918560_491151360.pth new file mode 100644 index 0000000000000000000000000000000000000000..716259903195cfd9cc9289b9a382726ac34918d5 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001918560_491151360.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:631246d668d4d196f457d043a4038f00ab7072f26170f880165a0a5ed418ee6a +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001932384_494690304.pth b/checkpoint_p0/milestones/checkpoint_001932384_494690304.pth new file mode 100644 index 0000000000000000000000000000000000000000..35d8111cd6006eda4ceeab61f626bdfb146d6a56 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001932384_494690304.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc98ff494d80e0c8a5719d5582181020037b8a77209d378689b80ea4856d8062 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001946272_498245632.pth b/checkpoint_p0/milestones/checkpoint_001946272_498245632.pth new file mode 100644 index 0000000000000000000000000000000000000000..742a9bae38378bf194904a624c78bbf5cf8500c0 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001946272_498245632.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:261924031294b6d26a5f7cd9ed806a233095b39276299539fc329ea5159ad5e6 +size 20723275 diff --git a/checkpoint_p0/milestones/checkpoint_001959200_501833728.pth b/checkpoint_p0/milestones/checkpoint_001959200_501833728.pth new file mode 100644 index 0000000000000000000000000000000000000000..2f384362aff1a3eab6e145c46936c50f95980079 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001959200_501833728.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8f8a4400eb64ed57fb511bbbf95be1d27ac118398f8418556dce2fa9a8a30db3 +size 20723275 diff --git a/checkpoint_p1/best_001951008_499458048_reward_166.580.pth b/checkpoint_p1/best_001951008_499458048_reward_166.580.pth new file mode 100644 index 0000000000000000000000000000000000000000..2a78edb37bb744a4f9760d07f2c0081bf90866c4 --- /dev/null +++ b/checkpoint_p1/best_001951008_499458048_reward_166.580.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d8232a64d0081d12a32697d069471e4f128e323c7feaf707e9e1c74a8436a25a +size 20722035 diff --git a/checkpoint_p1/checkpoint_001952704_499892224.pth b/checkpoint_p1/checkpoint_001952704_499892224.pth new file mode 100644 index 0000000000000000000000000000000000000000..bc91ae0c003cfa4a9475f156f3ccc967db663dad --- /dev/null +++ b/checkpoint_p1/checkpoint_001952704_499892224.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6010f937e9fe45c7f454153e62bac0ff4210664532c6ddb9eef2c9ab9bd967db +size 20722371 diff --git a/checkpoint_p1/checkpoint_001953184_500015104.pth b/checkpoint_p1/checkpoint_001953184_500015104.pth new file mode 100644 index 0000000000000000000000000000000000000000..0bf932759d9114483d9c6a2d61588ffe1ddf67d0 --- /dev/null +++ b/checkpoint_p1/checkpoint_001953184_500015104.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:62aa06e67f6c418e10b8b47d6d1774e0a9f10350aeb69c096271600acbd3012a +size 20722371 diff --git a/checkpoint_p1/milestones/checkpoint_000018400_4710400.pth b/checkpoint_p1/milestones/checkpoint_000018400_4710400.pth new file mode 100644 index 0000000000000000000000000000000000000000..d07b8b9edda7fdb906022d5dc525d10ae2e367c8 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000018400_4710400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0748edccba1043b8fb94eb75967af079164dde80894ed41de04b6e9300bb2583 +size 20723163 diff --git a/checkpoint_p1/milestones/checkpoint_000032128_8224768.pth b/checkpoint_p1/milestones/checkpoint_000032128_8224768.pth new file mode 100644 index 0000000000000000000000000000000000000000..afad76265a54cd7f3586608b9c4583f9271a65ba --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000032128_8224768.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc76f4c825c522e933e06cc9695a8bac05b427b44b924fa96edcb03d59649a2e +size 20723163 diff --git a/checkpoint_p1/milestones/checkpoint_000045888_11747328.pth b/checkpoint_p1/milestones/checkpoint_000045888_11747328.pth new file mode 100644 index 0000000000000000000000000000000000000000..068d10bbf107e3d536468eb417d7077dcc05991e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000045888_11747328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a894e59e9d7821d5068725cba97087bacb964ea7b32fcfca95f060a73ee8ec7 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000059648_15269888.pth b/checkpoint_p1/milestones/checkpoint_000059648_15269888.pth new file mode 100644 index 0000000000000000000000000000000000000000..8b77a51bb94aea2eb4b332569c926a21df570c92 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000059648_15269888.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4d03de6ebd7edd812070bfb173eb37cfcc96116f3df7497bd4bdf57457a3376b +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000073376_18784256.pth b/checkpoint_p1/milestones/checkpoint_000073376_18784256.pth new file mode 100644 index 0000000000000000000000000000000000000000..27f8b85505e94a10f5688eac575cb885ab69cdca --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000073376_18784256.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b908e6cd696900a8fc27f823305e26601522e64249410cb10e1cf6f76f394868 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000087104_22298624.pth b/checkpoint_p1/milestones/checkpoint_000087104_22298624.pth new file mode 100644 index 0000000000000000000000000000000000000000..00fb4a4c5054256159cb0997fc55e669f734db52 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000087104_22298624.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:646b85e271c72bc307cbf327fcf8f6051af08c8de34f26773c555f9db195d720 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000100928_25837568.pth b/checkpoint_p1/milestones/checkpoint_000100928_25837568.pth new file mode 100644 index 0000000000000000000000000000000000000000..006d124d08aeaa412ca5678fd87d4ee349e9ede5 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000100928_25837568.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d5cb8fda1c345a5cff8f2adc0d1f95be332203cd9afcda3b2ede71616ce2f47a +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000114656_29351936.pth b/checkpoint_p1/milestones/checkpoint_000114656_29351936.pth new file mode 100644 index 0000000000000000000000000000000000000000..e26f4281f3bfbb15b1cb13c9f693b634d864a550 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000114656_29351936.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc83b6e72693d7a603c2dfc2a5ec74087a48c0495faf9cce0d622b9ea6e88809 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000128416_32874496.pth b/checkpoint_p1/milestones/checkpoint_000128416_32874496.pth new file mode 100644 index 0000000000000000000000000000000000000000..66ff646d79a1fa065f83ea7df51cac51f7f23f60 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000128416_32874496.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c5b6bb7bedb6685426b48c4e7898995a632d89f5950a8c11956f4172c5a4d931 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000142208_36405248.pth b/checkpoint_p1/milestones/checkpoint_000142208_36405248.pth new file mode 100644 index 0000000000000000000000000000000000000000..8dfe9c5bb6d1477925729ad2579b76fd88f5612d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000142208_36405248.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6c4ca793e145c89c449750f252f48a305aff59a3b0b2b1a87130c6288f43e98 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000155872_39903232.pth b/checkpoint_p1/milestones/checkpoint_000155872_39903232.pth new file mode 100644 index 0000000000000000000000000000000000000000..7c25731f9200268a0dfaf843b56392fd8b02194c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000155872_39903232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5080a6fd485acd1ac8dc95bb5851f048686cf71f40920b74fd0018b79421cbd +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000169632_43425792.pth b/checkpoint_p1/milestones/checkpoint_000169632_43425792.pth new file mode 100644 index 0000000000000000000000000000000000000000..eb8e3dd5953b26ad07aaab3dc1f2e6f05928e8d6 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000169632_43425792.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a91f9420e77f16569488467e4742a98ca146b653ded091ae1ab9a2400c6866f +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000183360_46940160.pth b/checkpoint_p1/milestones/checkpoint_000183360_46940160.pth new file mode 100644 index 0000000000000000000000000000000000000000..e756fed3860cb9db97a6e2a22323273a428b9e50 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000183360_46940160.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3f5fcdcf8c2e6702c166031907f126e5c59f67a347dc5469f93933ca0476c04 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000197120_50462720.pth b/checkpoint_p1/milestones/checkpoint_000197120_50462720.pth new file mode 100644 index 0000000000000000000000000000000000000000..263a0c020d918df29134c9e1265a2f8aaaca2a82 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000197120_50462720.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:788935326599fdf574307e150903968a4af9ab170f70425e375e529f0b5d55b5 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000210912_53993472.pth b/checkpoint_p1/milestones/checkpoint_000210912_53993472.pth new file mode 100644 index 0000000000000000000000000000000000000000..95dabddcb0412a068cd26ae8f0f37d6eed3ee64f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000210912_53993472.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5a8baeb4a830ec2bf0d261a4b7a061271b15cc38adb4308f3b621014e8feef6 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000224672_57516032.pth b/checkpoint_p1/milestones/checkpoint_000224672_57516032.pth new file mode 100644 index 0000000000000000000000000000000000000000..43d009585aa9eee8d2f8e2e343f5110e7386ecbf --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000224672_57516032.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c710f83169967459cdab5621209c3ca7404fa569df6ab5d83b8fee038ec27cef +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000238432_61038592.pth b/checkpoint_p1/milestones/checkpoint_000238432_61038592.pth new file mode 100644 index 0000000000000000000000000000000000000000..68866ec7d7f720310b047daf3bbb3dc1328072c1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000238432_61038592.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:330ddd92f116c917ccadabc5d715e97dbf695c1153fa17914930969419a5ed63 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000252192_64561152.pth b/checkpoint_p1/milestones/checkpoint_000252192_64561152.pth new file mode 100644 index 0000000000000000000000000000000000000000..626fe62c11c5c987e59b19df1cb77a72a152e7cb --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000252192_64561152.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:61ee821ddf80bdb51d75faa1c170fe3906d421fc76dfe77e071beabbba85b84d +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000265952_68083712.pth b/checkpoint_p1/milestones/checkpoint_000265952_68083712.pth new file mode 100644 index 0000000000000000000000000000000000000000..bb9266ffe15252f3bcde1867bf2e3a4c1af2b3c8 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000265952_68083712.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f0a231dd1cdf5d6f3cf6c5f9acbe2830fb382aabf987457f8241b9059ed668db +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000279712_71606272.pth b/checkpoint_p1/milestones/checkpoint_000279712_71606272.pth new file mode 100644 index 0000000000000000000000000000000000000000..8003e74b8bf34cfede5191cbba1c5ade6c805947 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000279712_71606272.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ca5b58bea247f052619f12b74da75b65d7ed3aeb59b47b04370a95d35df7f29 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000293440_75120640.pth b/checkpoint_p1/milestones/checkpoint_000293440_75120640.pth new file mode 100644 index 0000000000000000000000000000000000000000..3b29f5dccfb4f270bcf76466e5bf0df596d24978 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000293440_75120640.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d58e6dbfc80fbfb661bc6bd13b2863e5a071e5e454cc51b35a6a053073ea92c9 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000307200_78643200.pth b/checkpoint_p1/milestones/checkpoint_000307200_78643200.pth new file mode 100644 index 0000000000000000000000000000000000000000..c53ed22a3f969a35d83ad225ee87aed524ea2db3 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000307200_78643200.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c5e5c1330e52f8cfec416470c2c78ba0978f32cb10dad4c5d493728110293a6 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000320928_82157568.pth b/checkpoint_p1/milestones/checkpoint_000320928_82157568.pth new file mode 100644 index 0000000000000000000000000000000000000000..7670cfef8fbaecd0ca0a6d25d285fbef9e2e9255 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000320928_82157568.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:542f70cbd73129bff41da0e9974b9e2e976cbb80a9dd4c2dfc5c998bce3cbe75 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000334592_85655552.pth b/checkpoint_p1/milestones/checkpoint_000334592_85655552.pth new file mode 100644 index 0000000000000000000000000000000000000000..397fb452be08d10a949a7e065af8f57fbd61531a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000334592_85655552.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f93ead87462d33e370025d362920a731beb8637335f91e76572c8d683e2e781e +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000348320_89169920.pth b/checkpoint_p1/milestones/checkpoint_000348320_89169920.pth new file mode 100644 index 0000000000000000000000000000000000000000..51143c87ee613e0acf453bbe9841fe02ac1d1677 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000348320_89169920.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de0815f3f8e73146af118c1d244f3113c851fa418385cadd5fca0b38d10abe54 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000362016_92676096.pth b/checkpoint_p1/milestones/checkpoint_000362016_92676096.pth new file mode 100644 index 0000000000000000000000000000000000000000..3d37e8001d70fa26696e1537531295622f78b46c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000362016_92676096.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ed4da1927e70ff32842e4b06811db67d5669b814f7f183cfbdd77eef5b8aad2 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000375744_96190464.pth b/checkpoint_p1/milestones/checkpoint_000375744_96190464.pth new file mode 100644 index 0000000000000000000000000000000000000000..fe8d8ad3b5d99c642fc0f19b06ece3f0e476aca6 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000375744_96190464.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4beb1ff73ed330fc0ee458841db967193e9106d450c8651b41d28d4d305a566 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000389472_99704832.pth b/checkpoint_p1/milestones/checkpoint_000389472_99704832.pth new file mode 100644 index 0000000000000000000000000000000000000000..f67ae59f506e535106c0e21c4ecbd11a1865a87c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000389472_99704832.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16db8dcd35fdac89514b08a1a60f2bd692b2046259148c86a86798b487a96001 +size 20723219 diff --git a/checkpoint_p1/milestones/checkpoint_000403200_103219200.pth b/checkpoint_p1/milestones/checkpoint_000403200_103219200.pth new file mode 100644 index 0000000000000000000000000000000000000000..f9406b344b6db24a141ef32e47d4eb855ef29670 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000403200_103219200.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:405eb195a7ec3ebc7a29b08363c95b66b27859936e63d38d1f58b4a9e4cb483a +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000416864_106717184.pth b/checkpoint_p1/milestones/checkpoint_000416864_106717184.pth new file mode 100644 index 0000000000000000000000000000000000000000..a21dac3987eeec0b56194a2d81ea71bb695e765f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000416864_106717184.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9a9e2a1a1e4f2cc8d64a1cf95e720cd449f46381ff66735f62229f6582a4d5c +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000430560_110223360.pth b/checkpoint_p1/milestones/checkpoint_000430560_110223360.pth new file mode 100644 index 0000000000000000000000000000000000000000..8f059ba92372de28324bd4f4e29e5f56dfcee465 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000430560_110223360.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:005b2e408f93d566921c452065b9bd0a5409b2ccc7a84184730d8be34c61e36b +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000444256_113729536.pth b/checkpoint_p1/milestones/checkpoint_000444256_113729536.pth new file mode 100644 index 0000000000000000000000000000000000000000..d4bfab3a44da3b3044ae0905e14370124cce95f1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000444256_113729536.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ddae3def2e1007c6fbbccd38de7f419a7496e919a049aacb945ea3cb2b5a879 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000457920_117227520.pth b/checkpoint_p1/milestones/checkpoint_000457920_117227520.pth new file mode 100644 index 0000000000000000000000000000000000000000..55f6f4d2e8d46c43b9b041482446a09a02cae8e0 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000457920_117227520.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e29a30dc20e5c2c764a10aed1d16aa379ca6b74dfd6004a49606d34ab48f53ce +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000471680_120750080.pth b/checkpoint_p1/milestones/checkpoint_000471680_120750080.pth new file mode 100644 index 0000000000000000000000000000000000000000..8fb23dd1ef11602f8df7d1a62401f3cc14f117af --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000471680_120750080.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:406c2e3fc0293d521139d790f7468b70da20b379f09d19aa8dc72590fab631dc +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000485376_124256256.pth b/checkpoint_p1/milestones/checkpoint_000485376_124256256.pth new file mode 100644 index 0000000000000000000000000000000000000000..a4e817cf2953c5adb7523e6197330ee07294c655 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000485376_124256256.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:708af7ff723f0951f9d1ecc77a631c6fca84f1a0eedbd997a23ec9e554da05a9 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000499104_127770624.pth b/checkpoint_p1/milestones/checkpoint_000499104_127770624.pth new file mode 100644 index 0000000000000000000000000000000000000000..80a0bf40aa98eba2a9188d5d49228b8b667d20db --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000499104_127770624.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8fd106b86ac84f98ab069b89578eab741ba05ee640975cfbe6f5403b9fcc167 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000512800_131276800.pth b/checkpoint_p1/milestones/checkpoint_000512800_131276800.pth new file mode 100644 index 0000000000000000000000000000000000000000..37218edcd22b2ac2f058474c8451d51f17e62ea6 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000512800_131276800.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e622acaa3a7f76ffebbcac72283732057443b558872ce7a3bbc89984bafe5de3 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000526496_134782976.pth b/checkpoint_p1/milestones/checkpoint_000526496_134782976.pth new file mode 100644 index 0000000000000000000000000000000000000000..20a23945265e78adf0f376b4598849e381234d1d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000526496_134782976.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a643821f9f05643b76d5db5bec13dc32db2bef7af5d10ffc9d5b1b701536a9d +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000540192_138289152.pth b/checkpoint_p1/milestones/checkpoint_000540192_138289152.pth new file mode 100644 index 0000000000000000000000000000000000000000..12ea6d8a6d19f4848cae32974101b486a2c69e79 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000540192_138289152.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5ccef77e2fd6f5b6bae7ccf45dd93e0cf5f296c89944362e901b2c633cb9deb +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000553888_141795328.pth b/checkpoint_p1/milestones/checkpoint_000553888_141795328.pth new file mode 100644 index 0000000000000000000000000000000000000000..519eebb8ebe1e1bbc08f927bafe2ff3ddd3826e1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000553888_141795328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5dbce026cb602bdc4f667529879b77d26e875e32aa46000e76091d877f7ec39a +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000567616_145309696.pth b/checkpoint_p1/milestones/checkpoint_000567616_145309696.pth new file mode 100644 index 0000000000000000000000000000000000000000..c8e15c0c214dca7213a3d8ae17bb1962604bb152 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000567616_145309696.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b40df0346a196a1fb9f985e4c4f5412c8ed7a65109535a5b7e0524e66a8412c2 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000581280_148807680.pth b/checkpoint_p1/milestones/checkpoint_000581280_148807680.pth new file mode 100644 index 0000000000000000000000000000000000000000..368e21131ad3a864d1b636ae950789a792ae7857 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000581280_148807680.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbe6b8ebf8185ef7b2d7a98a83820555b6c0e1569204d8ae867f54eab1ddee19 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000594976_152313856.pth b/checkpoint_p1/milestones/checkpoint_000594976_152313856.pth new file mode 100644 index 0000000000000000000000000000000000000000..03331bf8aac8f4bc71976b45eb610b56a4a86550 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000594976_152313856.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a3493ea3630f1c8b13562bbcbb3b459a413d60e6ba0486046e97cbc21a476753 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000608640_155811840.pth b/checkpoint_p1/milestones/checkpoint_000608640_155811840.pth new file mode 100644 index 0000000000000000000000000000000000000000..ddd76bc09011fc338036b56770ee883b3da6ce5a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000608640_155811840.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:24b111b23eb814b0347ef572a1aaf784e72facc9b1708bc9514daad7c6bfcb17 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000622368_159326208.pth b/checkpoint_p1/milestones/checkpoint_000622368_159326208.pth new file mode 100644 index 0000000000000000000000000000000000000000..e24b8fb7d7af2ea3216912c599bfcadc7bd0aa5c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000622368_159326208.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a4ff97601952ca51a6c38311690d5c7bbe3fc3ad7b07075001c54b93ea208a2 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000636032_162824192.pth b/checkpoint_p1/milestones/checkpoint_000636032_162824192.pth new file mode 100644 index 0000000000000000000000000000000000000000..693c0ef6e7e6d3b074dd0534dbd31e2fcb0862d5 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000636032_162824192.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff04383af95452b607e3daa25af95ce66bcdf9e102cfa546a66b94b68ef94a8d +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000649728_166330368.pth b/checkpoint_p1/milestones/checkpoint_000649728_166330368.pth new file mode 100644 index 0000000000000000000000000000000000000000..dc9ebcacc0993e1448e3f6ce578eaaa8d053f054 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000649728_166330368.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26b16a7c6d4756cdd43e418a79087b466c6546769c1b89d529674d0766a2ecfa +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000663360_169820160.pth b/checkpoint_p1/milestones/checkpoint_000663360_169820160.pth new file mode 100644 index 0000000000000000000000000000000000000000..0ae613e4f47debc551c5870190ecffa8d50e335b --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000663360_169820160.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d7cfeef7aee7ff6ce4c1bea6fc267ee6c687527fc495212b79d26151a85a95c5 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000676960_173301760.pth b/checkpoint_p1/milestones/checkpoint_000676960_173301760.pth new file mode 100644 index 0000000000000000000000000000000000000000..ecc29f83cc9835ee677edc7eae33183268b78b78 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000676960_173301760.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37fb7b94164aa86840f6bc9a609180d1b2e98dddac1ab90f525e2f913beb224c +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000690496_176766976.pth b/checkpoint_p1/milestones/checkpoint_000690496_176766976.pth new file mode 100644 index 0000000000000000000000000000000000000000..484eedb40114607f33533e374d938ff392579e05 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000690496_176766976.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:51514c3ef734fb9773e4821be3590d114e6c4133752b3e20ff571f94790ce86b +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000703616_180125696.pth b/checkpoint_p1/milestones/checkpoint_000703616_180125696.pth new file mode 100644 index 0000000000000000000000000000000000000000..2bd4afd413bf27a356396e5de6f51fa2f5330950 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000703616_180125696.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2d0706d540b7ef166477d1fbbea176ddb5b6510b83f41459ab8b4880ce59babf +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000716992_183549952.pth b/checkpoint_p1/milestones/checkpoint_000716992_183549952.pth new file mode 100644 index 0000000000000000000000000000000000000000..43f1256b3074dc5cdb7f8d97ab1cfccf39ed11b7 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000716992_183549952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cc70a327d27435c7bae6f4c2b8934a775ad86ed03cbda01a3b10fef326d44b5f +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000730560_187023360.pth b/checkpoint_p1/milestones/checkpoint_000730560_187023360.pth new file mode 100644 index 0000000000000000000000000000000000000000..b2f744b3432a72ffa61bebe502469c2a99e0d624 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000730560_187023360.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e97cb936fcdca27ad1f11457cde474a2ed5ec63631e7dd9043d7571a6cee62e1 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000744128_190496768.pth b/checkpoint_p1/milestones/checkpoint_000744128_190496768.pth new file mode 100644 index 0000000000000000000000000000000000000000..e7af49eb31fab09b2e4c61e689e28a7a40a0eb6a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000744128_190496768.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8958298b253925b893fc871db06116c4fe8c31df94fe63d540e0ec95ee55a954 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000757664_193961984.pth b/checkpoint_p1/milestones/checkpoint_000757664_193961984.pth new file mode 100644 index 0000000000000000000000000000000000000000..7e93854814941b8e5ab0dc57f8c231f09fdf54bb --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000757664_193961984.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d97139ffb2e282bc52fccecac7ba891ed3b155b1d177ce9b769b7176ce9b3cd6 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000771232_197435392.pth b/checkpoint_p1/milestones/checkpoint_000771232_197435392.pth new file mode 100644 index 0000000000000000000000000000000000000000..170cbb270adee0cb99c5c40f89c53200177addf3 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000771232_197435392.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2201ce326b919a1083614d790ab08bb50bbabe8e92108ab656560cb7f6ce0bd4 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000784832_200916992.pth b/checkpoint_p1/milestones/checkpoint_000784832_200916992.pth new file mode 100644 index 0000000000000000000000000000000000000000..5173e2fa9366daaec930c8ca72f8a776f19b5a0a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000784832_200916992.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:05a884905a54d5ccda3f7f1b557c21ff8ee3f977730d4cde001d5b3feeaa1d11 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000798464_204406784.pth b/checkpoint_p1/milestones/checkpoint_000798464_204406784.pth new file mode 100644 index 0000000000000000000000000000000000000000..70767f20c110ec70af5dab67ba34f8a762931079 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000798464_204406784.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0328ed1813850942e7784aaa66e0764132eea0e43b2cde6884f8e9173e75e546 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000812064_207888384.pth b/checkpoint_p1/milestones/checkpoint_000812064_207888384.pth new file mode 100644 index 0000000000000000000000000000000000000000..7c26dd293706fe05a0dc6d2ab51164997bd0e470 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000812064_207888384.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a21ee85d28299e0390ec67770302a4ee28532f075d07c16caf7136d0da9ddd4 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000825792_211402752.pth b/checkpoint_p1/milestones/checkpoint_000825792_211402752.pth new file mode 100644 index 0000000000000000000000000000000000000000..cf479789a23bf744ec55fe80472cd248d8a0c08d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000825792_211402752.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9e938ddec09442bc74ba569e5e98dc9b3ebdb1a8c71ea6db337759cb0093626c +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000839424_214892544.pth b/checkpoint_p1/milestones/checkpoint_000839424_214892544.pth new file mode 100644 index 0000000000000000000000000000000000000000..bf46e4590eb51faf41e18c77a65e7b3af37c9386 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000839424_214892544.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bcb0b4aca2a5a910f96a6a7895177d875335a3c1fc2fb263caaa695b94fb8502 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000853120_218398720.pth b/checkpoint_p1/milestones/checkpoint_000853120_218398720.pth new file mode 100644 index 0000000000000000000000000000000000000000..3e30e8095bd8ea549b1b5cdeb34a449264aceafd --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000853120_218398720.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0c411e93e2988003cd0378e56129aeaabb52a71eb3833efaf0fa7d90198fffe +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000866816_221904896.pth b/checkpoint_p1/milestones/checkpoint_000866816_221904896.pth new file mode 100644 index 0000000000000000000000000000000000000000..faa224ed9cffe83e36be41300a097df04e8381a6 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000866816_221904896.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd7714f899e7f7802883353a32b33fc6ddeff78b5db31988aa3feab91262a172 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000880512_225411072.pth b/checkpoint_p1/milestones/checkpoint_000880512_225411072.pth new file mode 100644 index 0000000000000000000000000000000000000000..c19c10e1e5bb52e8e768bc78cb47eed23dcb9c3f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000880512_225411072.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:390ccc453d48b88dd59435b91d07cafc87fd64d613ce4e3956abf036b9c85fd7 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000894208_228917248.pth b/checkpoint_p1/milestones/checkpoint_000894208_228917248.pth new file mode 100644 index 0000000000000000000000000000000000000000..f6d9164e86127b23e79582440e7f51a551d96fe5 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000894208_228917248.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55ab2ef30d789e075521bb7e07d06e125c77a537474617971fe58d595fcc82aa +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000907872_232415232.pth b/checkpoint_p1/milestones/checkpoint_000907872_232415232.pth new file mode 100644 index 0000000000000000000000000000000000000000..9b555075bd76956e06bf49c36e249d931508d7d9 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000907872_232415232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b8a81b0fa3b36f9a390cd4ee1abfc775617462aa97845fd9ab31f6c2f0ff006 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000921536_235913216.pth b/checkpoint_p1/milestones/checkpoint_000921536_235913216.pth new file mode 100644 index 0000000000000000000000000000000000000000..587243d53efa59684580a847f0c2293755d597a4 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000921536_235913216.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1bcadf226ac04a9065800938b53f983cf81af94bdb5e98136fc0a59f8a40fb29 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000935168_239403008.pth b/checkpoint_p1/milestones/checkpoint_000935168_239403008.pth new file mode 100644 index 0000000000000000000000000000000000000000..44d342f12a0204971b7b87f8c3f894536485e92c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000935168_239403008.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:70f184e747ff273b548e80de17c6830f9e3510f241c895528067ec2eb925abdf +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000948832_242900992.pth b/checkpoint_p1/milestones/checkpoint_000948832_242900992.pth new file mode 100644 index 0000000000000000000000000000000000000000..4037fcac0c78b9b6619a86ce8996718d904c7bd2 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000948832_242900992.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5372dc16da1306981538d6c65c6980bd8ce8e284dc18f45dc0e21cd3c6759092 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000962496_246398976.pth b/checkpoint_p1/milestones/checkpoint_000962496_246398976.pth new file mode 100644 index 0000000000000000000000000000000000000000..25f491323dc24c2ed858924a74fac9399544406f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000962496_246398976.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7340c8b42a0277423edcd85cf008b12123321ce37c13f82523003acfc1a582a6 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000976128_249888768.pth b/checkpoint_p1/milestones/checkpoint_000976128_249888768.pth new file mode 100644 index 0000000000000000000000000000000000000000..c46f353bab9cf96dd9ae2cc62530da8f78e21f6f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000976128_249888768.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3563d6171c4fab38f533abdd221d871628746a9d2ceb0f1d21f56037f1173f26 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_000989760_253378560.pth b/checkpoint_p1/milestones/checkpoint_000989760_253378560.pth new file mode 100644 index 0000000000000000000000000000000000000000..f667a32e0e434e250deb1eb44bd0be8e2ae437c1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000989760_253378560.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c324dfb503b6cdbe17bf5f0949f81cc3b9b72d9ce299c301f225469c48344da +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001003488_256892928.pth b/checkpoint_p1/milestones/checkpoint_001003488_256892928.pth new file mode 100644 index 0000000000000000000000000000000000000000..5e4b38ed7408e051e70f8c9d05885442b657c3a5 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001003488_256892928.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:76bee6c29108fc733dc97893d54677a92f4b581fd345f5ccace55a262bff6f13 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001017184_260399104.pth b/checkpoint_p1/milestones/checkpoint_001017184_260399104.pth new file mode 100644 index 0000000000000000000000000000000000000000..0f2ebba6ca1af7e738d115a91b4b3d62677d190c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001017184_260399104.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cd29a0f7f46e4e95bb6aeb4f72e215c2b5bf0209a00932f88299006125887f20 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001030848_263897088.pth b/checkpoint_p1/milestones/checkpoint_001030848_263897088.pth new file mode 100644 index 0000000000000000000000000000000000000000..5028548652db93149a12b4a8a590cf80e5c2d15c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001030848_263897088.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f0b9cc9307412bb933d0436ee754cd31623e4859e777697bacbfbe23d290b18 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001044480_267386880.pth b/checkpoint_p1/milestones/checkpoint_001044480_267386880.pth new file mode 100644 index 0000000000000000000000000000000000000000..25b203c0123564c76fb9a2c6f7e44efe0cf5fc1e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001044480_267386880.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8f8b31f386d1ffa41ef271047bb417aa21a12d17cba2a9aeacfacef063f186e2 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001058176_270893056.pth b/checkpoint_p1/milestones/checkpoint_001058176_270893056.pth new file mode 100644 index 0000000000000000000000000000000000000000..c81a5013717c3902e9454af043952431c26cfa04 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001058176_270893056.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9bf3590d087eb9332cccd25077c25353c5d5b766b52d29a155b71e00f43222b0 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001071776_274374656.pth b/checkpoint_p1/milestones/checkpoint_001071776_274374656.pth new file mode 100644 index 0000000000000000000000000000000000000000..bf3f71449202d4136b3d4ede29d71289772c5af8 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001071776_274374656.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea72ddd7ff40a035a7afa10e60a3bd7273f851a46e2f9b19b61d22b098b7a86f +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001085120_277790720.pth b/checkpoint_p1/milestones/checkpoint_001085120_277790720.pth new file mode 100644 index 0000000000000000000000000000000000000000..302c500ef388aa1cd7c93b891416bfad7dc4fccd --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001085120_277790720.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15e268bae6546121af5dc9ea14cbb9c5884ca88fede204fa154c34e833921e8f +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001098624_281247744.pth b/checkpoint_p1/milestones/checkpoint_001098624_281247744.pth new file mode 100644 index 0000000000000000000000000000000000000000..8247cfe3bb729dedfcbfeac15a78be343adfa9fe --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001098624_281247744.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a99a4cfbd7da8684b651af24db0a4a61824a26091cac697089fe9b124ae85700 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001112192_284721152.pth b/checkpoint_p1/milestones/checkpoint_001112192_284721152.pth new file mode 100644 index 0000000000000000000000000000000000000000..be53398bef551842cdc8d089ce96ed66129a19b3 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001112192_284721152.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c77be6c619328bac3e5dc999c1a00b1c2c27f202577d07e410a70a9a0274d258 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001125728_288186368.pth b/checkpoint_p1/milestones/checkpoint_001125728_288186368.pth new file mode 100644 index 0000000000000000000000000000000000000000..8f0fc5289bc7b8942f879cce2b88156f38ae06be --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001125728_288186368.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a3f859f74b3ac61ace68066eae9f44f3fe107a7bdf1ab17258ba35fc2120c1e +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001139296_291659776.pth b/checkpoint_p1/milestones/checkpoint_001139296_291659776.pth new file mode 100644 index 0000000000000000000000000000000000000000..6fad49a734f8f300a0b70019b52e78714f731ea3 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001139296_291659776.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:067abeee9e8da1003ba5c1e09d7c4312f1f0d6b1867facc1d0b62b3a4ed9018f +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001152864_295133184.pth b/checkpoint_p1/milestones/checkpoint_001152864_295133184.pth new file mode 100644 index 0000000000000000000000000000000000000000..c7cd55e9db1b27fcd04e00ef7ca673b2fc322fd1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001152864_295133184.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:446faf4d56babeb88042cac7780568346ffa55388950dcaac478f9d843a61f78 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001166080_298516480.pth b/checkpoint_p1/milestones/checkpoint_001166080_298516480.pth new file mode 100644 index 0000000000000000000000000000000000000000..fcba1a8ced9273e1da240fbf85e91bb261f22896 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001166080_298516480.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e3872644b780d17d931854c8f19000ffd373858bfe4982a6f54707c785b41d77 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001179616_301981696.pth b/checkpoint_p1/milestones/checkpoint_001179616_301981696.pth new file mode 100644 index 0000000000000000000000000000000000000000..e97437ec47b09a0ab5cb1a4e4eeb8149f3b7d512 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001179616_301981696.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7ec2baefbe1c43b2d798f5dd3e4480f5321d6008285b8329f0597bdfada389b +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001193088_305430528.pth b/checkpoint_p1/milestones/checkpoint_001193088_305430528.pth new file mode 100644 index 0000000000000000000000000000000000000000..e1a5b0027734432fdfb2c3eb6afd733ef4785b44 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001193088_305430528.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6bd112fb6627a34b1083cb611ad6bc28e773c5abaab734c7563ea36818f8a879 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001206560_308879360.pth b/checkpoint_p1/milestones/checkpoint_001206560_308879360.pth new file mode 100644 index 0000000000000000000000000000000000000000..f9e1b7c003e02e95d7d7a100920f9c4266688b7c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001206560_308879360.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b3f759976c411e917e1d2ce0367b7c49c408a359cb5e1f5b98ee4104e69baa18 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001220032_312328192.pth b/checkpoint_p1/milestones/checkpoint_001220032_312328192.pth new file mode 100644 index 0000000000000000000000000000000000000000..2e0f2c7fb33deecb29d96d32cd0623f6c17e368d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001220032_312328192.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea63c583adbbc8d95f372b44c13ff834bcc2a47403ea34425a783b4b3efd5581 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001233568_315793408.pth b/checkpoint_p1/milestones/checkpoint_001233568_315793408.pth new file mode 100644 index 0000000000000000000000000000000000000000..7de4dd93c735bf8e44cb2043649ada108825a78a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001233568_315793408.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48a5df61f2b91584397a5b346520eb5f28632a88e722a557de76cb2c4b586d60 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001247040_319242240.pth b/checkpoint_p1/milestones/checkpoint_001247040_319242240.pth new file mode 100644 index 0000000000000000000000000000000000000000..49c6759a6116ee4fc543e3f9c249d0ad62b7b8c3 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001247040_319242240.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9b15c9939e93a6ea40147e013554964f4bd4e78257c8eb4713a7af157e49837 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001260576_322707456.pth b/checkpoint_p1/milestones/checkpoint_001260576_322707456.pth new file mode 100644 index 0000000000000000000000000000000000000000..bf923623d8557e7cac0f35a68029e78210139049 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001260576_322707456.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81582206c3e3c2d54fc452057f7965ca290834c6e09d1513430cb380dda3407d +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001274080_326164480.pth b/checkpoint_p1/milestones/checkpoint_001274080_326164480.pth new file mode 100644 index 0000000000000000000000000000000000000000..29db4f780dec6f8768ac0a6e9bad6772e14c2041 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001274080_326164480.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e4022d551757961e89c05de2b0fd4d8eeaa03da9b93e37eef25133eef4fa7a02 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001287552_329613312.pth b/checkpoint_p1/milestones/checkpoint_001287552_329613312.pth new file mode 100644 index 0000000000000000000000000000000000000000..791da4f3949252ddaedca50eb1d4b9311b4ae078 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001287552_329613312.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:421eb0ced6a52d049f934eb4279dade1958dfcb7b62b81d2cd81c80dfd21fbb4 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001301024_333062144.pth b/checkpoint_p1/milestones/checkpoint_001301024_333062144.pth new file mode 100644 index 0000000000000000000000000000000000000000..0d1e6b8bef8527667e8035ad7428a8cf912bbcc4 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001301024_333062144.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3b5f4276b3fda956d038b15e7ac2a4d57e999361a8abdc5dfa6af40d969ded0 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001314496_336510976.pth b/checkpoint_p1/milestones/checkpoint_001314496_336510976.pth new file mode 100644 index 0000000000000000000000000000000000000000..d849badd2902224dbe0ddabb4dbf7a1d021d31a4 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001314496_336510976.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:35de0c652c512f49578b1b1dc1b21b7609624805fb500b6e7c3a1f982f4c88e8 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001328000_339968000.pth b/checkpoint_p1/milestones/checkpoint_001328000_339968000.pth new file mode 100644 index 0000000000000000000000000000000000000000..0f5ea6216ad16bc15d8ebcd9947f1d3f52702274 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001328000_339968000.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8abca9cc535d38fa8f3dc481edb77c8a4ba230575f5818d20d9869fa985b4a1b +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001341504_343425024.pth b/checkpoint_p1/milestones/checkpoint_001341504_343425024.pth new file mode 100644 index 0000000000000000000000000000000000000000..eb4c4cb49152873d69b2fa150ddb3721d9a23731 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001341504_343425024.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f23aa53239bb1279b8565a52cbd93fbbf17505f892ccbeb45b1fc2851b2db40d +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001355008_346882048.pth b/checkpoint_p1/milestones/checkpoint_001355008_346882048.pth new file mode 100644 index 0000000000000000000000000000000000000000..55e0f1f12597c08d209c901b04d64cd476497003 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001355008_346882048.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec139042c0a209a276da5843e25df45dc571b5c29c9d8bd714dcabd0cb63e2ba +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001368544_350347264.pth b/checkpoint_p1/milestones/checkpoint_001368544_350347264.pth new file mode 100644 index 0000000000000000000000000000000000000000..5a9539cc83fe566cb59216bd9f87dfb6c4e2ba72 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001368544_350347264.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc1350be921c591afc1488dceaf50f56399e918410b5bc5cdedbbcdc270be26a +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001382112_353820672.pth b/checkpoint_p1/milestones/checkpoint_001382112_353820672.pth new file mode 100644 index 0000000000000000000000000000000000000000..04d3b38f4138839b9d4b9cfd72c78a36c4c6ab42 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001382112_353820672.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fee7c3b5ddf9d90ec492d11e0caf21671c48c3d0e5a6beb01812088fd61774a4 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001395616_357277696.pth b/checkpoint_p1/milestones/checkpoint_001395616_357277696.pth new file mode 100644 index 0000000000000000000000000000000000000000..2970dc3ba020a31da674dda5869f6a71f03932da --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001395616_357277696.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:42ef574f33e70955e0ced22a94f0f0200db707226d3c8523351d9629b8d2d17d +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001409184_360751104.pth b/checkpoint_p1/milestones/checkpoint_001409184_360751104.pth new file mode 100644 index 0000000000000000000000000000000000000000..711bf255b02f92c69a4626a11adc10ed0c5571dd --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001409184_360751104.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:911838099dcee3463de9806caab6cf90f7dff469b59addbd8aac1f5daae09042 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001422752_364224512.pth b/checkpoint_p1/milestones/checkpoint_001422752_364224512.pth new file mode 100644 index 0000000000000000000000000000000000000000..6c19bb98e8d4bcfa651029012277e207dd0d9d61 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001422752_364224512.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d57842d7b1e6d88c2e5c0b055cdf2c0c2cd4f90dbf43a3c8621c315cd59a7b3 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001436352_367706112.pth b/checkpoint_p1/milestones/checkpoint_001436352_367706112.pth new file mode 100644 index 0000000000000000000000000000000000000000..b7948805eae2a0d754cdc54356f4cc7f1fdb090e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001436352_367706112.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed39303b9eb6c9337860bed58d9d99ab259b94cc952ede1a58f7fd468a1c3c28 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001449888_371171328.pth b/checkpoint_p1/milestones/checkpoint_001449888_371171328.pth new file mode 100644 index 0000000000000000000000000000000000000000..99610e5b2e44f68d524a0f408b1b0a77a75d936c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001449888_371171328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f498ee24316ddd3ccdb8bb7019fb65cdc5fed212e9ec5dfc5d5dea86393420f2 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001463392_374628352.pth b/checkpoint_p1/milestones/checkpoint_001463392_374628352.pth new file mode 100644 index 0000000000000000000000000000000000000000..1566dc836a0928d54ef25f758ad82d45635a4508 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001463392_374628352.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b08ab50a0fc3d8e7b0686f6508cbd98677b5c6a08cf1045eab5ec3b71a95030f +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001476992_378109952.pth b/checkpoint_p1/milestones/checkpoint_001476992_378109952.pth new file mode 100644 index 0000000000000000000000000000000000000000..a020d976a2c18ac45bb983d0e8e6eb56122e2118 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001476992_378109952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:914bdc3945d8277d2cb845df16cf5a8b5e1e46922e2209049a2fa9a576ac10fb +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001490496_381566976.pth b/checkpoint_p1/milestones/checkpoint_001490496_381566976.pth new file mode 100644 index 0000000000000000000000000000000000000000..cf975f248255d52fe9a01113772a3949b408fb49 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001490496_381566976.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ff9610412d65908b3bc41020948ef276e74715fbc4151e172e54273f16e96a1 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001504096_385048576.pth b/checkpoint_p1/milestones/checkpoint_001504096_385048576.pth new file mode 100644 index 0000000000000000000000000000000000000000..5befeb5d469693289f6ddc67ce6564218edf3d2d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001504096_385048576.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e794de593e664e984128822465908bad8a0335656984df32eb9e437ca881ed3 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001517696_388530176.pth b/checkpoint_p1/milestones/checkpoint_001517696_388530176.pth new file mode 100644 index 0000000000000000000000000000000000000000..b1e35861684dc010e99c1a2f3777bd8b168054c0 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001517696_388530176.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a24db520cdf833ea77c04e52e872fdd38c914c261e6ca4280c5035a264f6a541 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001531296_392011776.pth b/checkpoint_p1/milestones/checkpoint_001531296_392011776.pth new file mode 100644 index 0000000000000000000000000000000000000000..ef7937afca57cb3816d25b3156d6087c93778474 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001531296_392011776.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:95157025ed4ddd7eebeb4028fae4bfd31958d0b595ed5dbbee5a3c9aee2e90bb +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001544960_395509760.pth b/checkpoint_p1/milestones/checkpoint_001544960_395509760.pth new file mode 100644 index 0000000000000000000000000000000000000000..111085d380b69c6f93a98b1b12c452943923bac9 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001544960_395509760.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2987a0d09119677b95e9caf8678bed072d45b29eba7aa728af05a877400da8b +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001558624_399007744.pth b/checkpoint_p1/milestones/checkpoint_001558624_399007744.pth new file mode 100644 index 0000000000000000000000000000000000000000..1f7e78ddc14aa99cc24e5cc32ab62ea593899f56 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001558624_399007744.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0f424b9519fa20af88386c02e089fcd12e06c436b85a0d98efcb9f658c0a801c +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001572288_402505728.pth b/checkpoint_p1/milestones/checkpoint_001572288_402505728.pth new file mode 100644 index 0000000000000000000000000000000000000000..02f7cf50175156e64f156af766689ab2c0182782 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001572288_402505728.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1927d2a3b6293cdda7e67a353c2e5a988272fc845e98ee24f12ce8cb36f5f949 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001585952_406003712.pth b/checkpoint_p1/milestones/checkpoint_001585952_406003712.pth new file mode 100644 index 0000000000000000000000000000000000000000..9d813bbbb60469719dad9a83ab7b1f7bd4489d71 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001585952_406003712.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b9091475be37066fd72b6898536bd8879bd6937a8a59916a6de3f5c5ae8747c +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001599552_409485312.pth b/checkpoint_p1/milestones/checkpoint_001599552_409485312.pth new file mode 100644 index 0000000000000000000000000000000000000000..8b8fb2cedbd4e5b18939b67354be843bc07d3047 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001599552_409485312.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c5ec02d5d261afb1f08a41d23b30cf076f2ee3beba9df9501ffd778bb13ab0ee +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001613280_412999680.pth b/checkpoint_p1/milestones/checkpoint_001613280_412999680.pth new file mode 100644 index 0000000000000000000000000000000000000000..d2fe9b4a365a71e18a3f6209992c867c0d2007eb --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001613280_412999680.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d17879fc21b6e22d005185508685153cad350903f71da6e127fcd80b15161d01 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001626944_416497664.pth b/checkpoint_p1/milestones/checkpoint_001626944_416497664.pth new file mode 100644 index 0000000000000000000000000000000000000000..c115a12d7958608d0c7499dbe3d3a823abfdfe8a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001626944_416497664.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55ec03258279b64d477dfca6b70e91af801ec8422eb35436046179400aec1086 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001640672_420012032.pth b/checkpoint_p1/milestones/checkpoint_001640672_420012032.pth new file mode 100644 index 0000000000000000000000000000000000000000..fdcabc6df65f2ee8b4bb85d16b828ace70a38cca --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001640672_420012032.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8ad6d3e77a9d21bb0421aa6e18e2665eec7d60d326dccef38879f9286b0dd73 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001654240_423485440.pth b/checkpoint_p1/milestones/checkpoint_001654240_423485440.pth new file mode 100644 index 0000000000000000000000000000000000000000..c0538b74288a603ba8d41fe6eb80feefd122bc3d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001654240_423485440.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:21df2f38066707b34dbe9d7e571d59d6cfed396bb993bd427403dd74a6db5dfc +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001667872_426975232.pth b/checkpoint_p1/milestones/checkpoint_001667872_426975232.pth new file mode 100644 index 0000000000000000000000000000000000000000..ea157663ed1c75df0d7ff4c3f58ea35d03f95981 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001667872_426975232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f6f4f50f7f2f4a3f3ddcf5e1c85b4f799305989377535e2491f8d3805340efb +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001681120_430366720.pth b/checkpoint_p1/milestones/checkpoint_001681120_430366720.pth new file mode 100644 index 0000000000000000000000000000000000000000..6ce41e13f393f3b7c547d1a049c462e63c9fc92d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001681120_430366720.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cca6512b1b0e8712f583a679f8a719e50bea2dd68e2c13669bace21b3f810c39 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001694592_433815552.pth b/checkpoint_p1/milestones/checkpoint_001694592_433815552.pth new file mode 100644 index 0000000000000000000000000000000000000000..9f7b48f8d355a7fed1405b270af88f2e5a71f858 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001694592_433815552.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8c79e2a533b04ac897539d4e0e67953e1d481b473e34ea7b0a43708944ca9ac +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001708256_437313536.pth b/checkpoint_p1/milestones/checkpoint_001708256_437313536.pth new file mode 100644 index 0000000000000000000000000000000000000000..519174bb3c96ba6e1a5602cd91cbac109eefc593 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001708256_437313536.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2668b1072c56c75dac5edc7dbc5a1724de2078f96b594a485193de899d3874ff +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001721888_440803328.pth b/checkpoint_p1/milestones/checkpoint_001721888_440803328.pth new file mode 100644 index 0000000000000000000000000000000000000000..c17621a1154279d253938a4e311dd16f7fc4f5b7 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001721888_440803328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:03724b093e6017a3cca48f52654f81404c3f9c8957d27d277b6333477334e855 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001735584_444309504.pth b/checkpoint_p1/milestones/checkpoint_001735584_444309504.pth new file mode 100644 index 0000000000000000000000000000000000000000..ef6c0dc3cd5b4212b8be124c9091fff84a696a4c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001735584_444309504.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7bffbe87df21167d6e59c4f732b6976ea22599b0607d503c4924d1ac49216338 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001749248_447807488.pth b/checkpoint_p1/milestones/checkpoint_001749248_447807488.pth new file mode 100644 index 0000000000000000000000000000000000000000..8e0a3c41e92bd6a455d5e876bd31c713e6ad2b4a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001749248_447807488.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:156fcd87159fe18971b66016d4d52aa3b900c66650692941ea7ba339e9433a8d +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001762912_451305472.pth b/checkpoint_p1/milestones/checkpoint_001762912_451305472.pth new file mode 100644 index 0000000000000000000000000000000000000000..654f75cec03e7bf920f24e2faaf8b62cdad00fdd --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001762912_451305472.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04cd5c0a1f4d7cc18efba7f84a859703945574ca8a6aa571f027cef1f03daa88 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001776576_454803456.pth b/checkpoint_p1/milestones/checkpoint_001776576_454803456.pth new file mode 100644 index 0000000000000000000000000000000000000000..76eaf76543fa384a82fe3d905bad21afc158310e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001776576_454803456.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8556026bcf165030ec3e74cf04f7bff5914ed158aeddf59ca9a1cb0bda8a899 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001790240_458301440.pth b/checkpoint_p1/milestones/checkpoint_001790240_458301440.pth new file mode 100644 index 0000000000000000000000000000000000000000..03a1a189f4e642a2ac10c037d3c4254d93951279 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001790240_458301440.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d4506b67ab2884b747317ae062d50a8a4abe17977353e1b51990b35d8494f33c +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001803872_461791232.pth b/checkpoint_p1/milestones/checkpoint_001803872_461791232.pth new file mode 100644 index 0000000000000000000000000000000000000000..594d88802f2cc2baf0fdf99f005ec97a6fc9b190 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001803872_461791232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f20de11e7465c67bb0fd3ba7d006df3cd2e009b2be4a80e7789b1d8da3ec7360 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001817504_465281024.pth b/checkpoint_p1/milestones/checkpoint_001817504_465281024.pth new file mode 100644 index 0000000000000000000000000000000000000000..27d35b9e62299eb28492bd6b0614f0f87190bc9f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001817504_465281024.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be48cb7d8b1417dc9d71861fadc1ba472203baaa8df9a32c235d59bccdbb12e2 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001831136_468770816.pth b/checkpoint_p1/milestones/checkpoint_001831136_468770816.pth new file mode 100644 index 0000000000000000000000000000000000000000..4a4a047b1360d383372a7127bb76e0f2df01e218 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001831136_468770816.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fd7a216b92e44d36c40f351ecadfaf0620265faa6c352aa29532155e493cdea5 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001844832_472276992.pth b/checkpoint_p1/milestones/checkpoint_001844832_472276992.pth new file mode 100644 index 0000000000000000000000000000000000000000..cb4c2ed313050f3e2657c78f177cccbacdb308f9 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001844832_472276992.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ca78627e6a80e83e98fa5e1a6e324e6d2f3d0bb9d6b8f3b16006ab42ae01f82 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001858560_475791360.pth b/checkpoint_p1/milestones/checkpoint_001858560_475791360.pth new file mode 100644 index 0000000000000000000000000000000000000000..a1023b5ce481482a0064600025d292f38269be2f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001858560_475791360.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89d21582a8a3d48faa9d933210ef801c3d1a63e8eee2004b7bc05e22282ec07d +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001872224_479289344.pth b/checkpoint_p1/milestones/checkpoint_001872224_479289344.pth new file mode 100644 index 0000000000000000000000000000000000000000..7068f6297c4ad4ef2aff678e193671ce9a3b4b5c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001872224_479289344.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd9404bc0a3d55326d45031e6e687f749c2482d1e849996e0e35aebb94bf91f7 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001885856_482779136.pth b/checkpoint_p1/milestones/checkpoint_001885856_482779136.pth new file mode 100644 index 0000000000000000000000000000000000000000..64abf8869534d6ae609612532184891454cabc17 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001885856_482779136.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3aa50afa54254eb01448202df33017ecc19f2125cb6374cc55e40ec9c0fc9c1e +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001899488_486268928.pth b/checkpoint_p1/milestones/checkpoint_001899488_486268928.pth new file mode 100644 index 0000000000000000000000000000000000000000..3a7c2e38e7eb1251abfc304178f27fb01d734de2 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001899488_486268928.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cd961c0626303fe893f1beebe45892075c108f16b0fcb2d7e001ceb396945de2 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001913184_489775104.pth b/checkpoint_p1/milestones/checkpoint_001913184_489775104.pth new file mode 100644 index 0000000000000000000000000000000000000000..ca5c0973feab75c364fbf5a209bac7e5aea77a92 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001913184_489775104.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4357d8492d91415bdb69de9f029e4782ad75de73eb9e0c71ee9436727c528077 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001926848_493273088.pth b/checkpoint_p1/milestones/checkpoint_001926848_493273088.pth new file mode 100644 index 0000000000000000000000000000000000000000..3d3ae057e4ab30255017d0a45753751e4187c160 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001926848_493273088.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:97a108cda653f2bd954df51177fa48648a609d677b43960d5794bd43633ebf98 +size 20723275 diff --git a/checkpoint_p1/milestones/checkpoint_001940448_496754688.pth b/checkpoint_p1/milestones/checkpoint_001940448_496754688.pth new file mode 100644 index 0000000000000000000000000000000000000000..0becc9f2f1facf18da845c19e664ef634c3cb244 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001940448_496754688.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ca364e6eed26ea85ad46e512b929416541ba3f486949163a209b0f002e952f0b +size 20723275 diff --git a/config.json b/config.json index 5513bf728bab0e62880c5bc0b0206b0c90c34448..5e91820f8e5448a4901196ead332114f46bd9f32 100644 --- a/config.json +++ b/config.json @@ -4,7 +4,7 @@ "env": "atari_spaceinvaders", "experiment": "atari_spaceinvaders_APPO", "train_dir": "./train_atari", - "restart_behavior": "restart", + "restart_behavior": "resume", "device": "gpu", "seed": 1234, "num_policies": 2, @@ -12,11 +12,11 @@ "serial_mode": false, "batched_sampling": true, "num_batches_to_accumulate": 2, - "worker_num_splits": 1, + "worker_num_splits": 2, "policy_workers_per_policy": 1, "max_policy_lag": 1000, "num_workers": 16, - "num_envs_per_worker": 2, + "num_envs_per_worker": 8, "batch_size": 1024, "num_batches_per_epoch": 8, "num_epochs": 4, @@ -64,10 +64,10 @@ "experiment_summaries_interval": 3, "flush_summaries_interval": 30, "stats_avg": 100, - "summaries_use_frameskip": true, + "summaries_use_frameskip": false, "heartbeat_interval": 10, "heartbeat_reporting_interval": 60, - "train_for_env_steps": 100000000, + "train_for_env_steps": 500000000, "train_for_seconds": 10000000000, "save_every_sec": 120, "keep_checkpoints": 2, @@ -124,28 +124,30 @@ "pbt_target_objective": "true_objective", "pbt_perturb_min": 1.1, "pbt_perturb_max": 1.5, - "command_line": "--algo=APPO --env=atari_spaceinvaders --experiment=atari_spaceinvaders_APPO --num_policies=2 --restart_behavior=restart --train_dir=./train_atari --train_for_env_steps=100000000 --seed=1234 --num_workers=16 --num_envs_per_worker=2 --num_batches_per_epoch=8 --async_rl=true --batched_sampling=true --batch_size=1024 --max_grad_norm=0 --learning_rate=0.0003033891184 --heartbeat_interval=10 --heartbeat_reporting_interval=60 --save_milestones_sec=1200 --num_epochs=4 --exploration_loss_coeff=0.0004677351413 --with_wandb=true --wandb_user=matt-stammers --wandb_project=atari_APPO --wandb_group=atari_spaceinvaders --wandb_job_type=SF --wandb_tags=atari", + "command_line": "--algo=APPO --env=atari_spaceinvaders --experiment=atari_spaceinvaders_APPO --num_policies=2 --restart_behavior=resume --train_dir=./train_atari --train_for_env_steps=500000000 --seed=1234 --num_workers=16 --num_envs_per_worker=8 --num_batches_per_epoch=8 --worker_num_splits=2 --async_rl=true --batched_sampling=true --batch_size=1024 --max_grad_norm=0 --learning_rate=0.0003033891184 --heartbeat_interval=10 --heartbeat_reporting_interval=60 --save_milestones_sec=1200 --num_epochs=4 --exploration_loss_coeff=0.0004677351413 --summaries_use_frameskip=False --with_wandb=true --wandb_user=matt-stammers --wandb_project=atari_APPO --wandb_group=atari_spaceinvaders --wandb_job_type=SF --wandb_tags=atari", "cli_args": { "algo": "APPO", "env": "atari_spaceinvaders", "experiment": "atari_spaceinvaders_APPO", "train_dir": "./train_atari", - "restart_behavior": "restart", + "restart_behavior": "resume", "seed": 1234, "num_policies": 2, "async_rl": true, "batched_sampling": true, + "worker_num_splits": 2, "num_workers": 16, - "num_envs_per_worker": 2, + "num_envs_per_worker": 8, "batch_size": 1024, "num_batches_per_epoch": 8, "num_epochs": 4, "exploration_loss_coeff": 0.0004677351413, "max_grad_norm": 0.0, "learning_rate": 0.0003033891184, + "summaries_use_frameskip": false, "heartbeat_interval": 10, "heartbeat_reporting_interval": 60, - "train_for_env_steps": 100000000, + "train_for_env_steps": 500000000, "save_milestones_sec": 1200, "with_wandb": true, "wandb_user": "matt-stammers", @@ -158,5 +160,5 @@ }, "git_hash": "5fff97c2f535da5987d358cdbe6927cccd43621e", "git_repo_name": "not a git repository", - "wandb_unique_id": "atari_spaceinvaders_APPO_20231015_144740_549444" + "wandb_unique_id": "atari_spaceinvaders_APPO_20231201_041318_464252" } \ No newline at end of file diff --git a/git.diff b/git.diff index 960bf7b013feefe7b56842bffdcf222f0bdf7dbd..f2014ff0d08b4ad19d4c267f4668e0df6f312c93 100644 --- a/git.diff +++ b/git.diff @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:3357904f421d3f4924836316b1741bf64d5dd0e807d5e80ac07059b4c52a7008 -size 14426734 +oid sha256:de4fecb91705490b8f6f89418f0c59ae52b7bc523a512f22d64b0d2006864d31 +size 380928 diff --git a/replay.mp4 b/replay.mp4 index 16763a2554e341ec5a3bc8bc4cb2969c938c1c34..6b7aa1a53baab03e3aed32f7be6954ee4b59b782 100644 --- a/replay.mp4 +++ b/replay.mp4 @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:c5c4a938cb417a890c8fb2537e0a9b2448158d869ad51bfa0aff673ee01b0b98 -size 5257062 +oid sha256:2c371fad8aca616da5e3cfabb8e4ce2cb0c612f1ac69fdf6eabba7bc2f391551 +size 45378018 diff --git a/sf_log.txt b/sf_log.txt index 73ae7c6ae076cd294d6b731bc2fac7cbb0da978b..7c4cdcf839545331557ba0817ccb7f4c37b3b1c0 100644 --- a/sf_log.txt +++ b/sf_log.txt @@ -1,26256 +1,3 @@ -[2023-10-15 14:47:47,364][51532] Saving configuration to ./train_atari/atari_spaceinvaders_APPO/config.json... -[2023-10-15 14:47:47,680][51532] Rollout worker 0 uses device cpu -[2023-10-15 14:47:47,681][51532] Rollout worker 1 uses device cpu -[2023-10-15 14:47:47,682][51532] Rollout worker 2 uses device cpu -[2023-10-15 14:47:47,682][51532] Rollout worker 3 uses device cpu -[2023-10-15 14:47:47,683][51532] Rollout worker 4 uses device cpu -[2023-10-15 14:47:47,683][51532] Rollout worker 5 uses device cpu -[2023-10-15 14:47:47,684][51532] Rollout worker 6 uses device cpu -[2023-10-15 14:47:47,684][51532] Rollout worker 7 uses device cpu -[2023-10-15 14:47:47,685][51532] Rollout worker 8 uses device cpu -[2023-10-15 14:47:47,685][51532] Rollout worker 9 uses device cpu -[2023-10-15 14:47:47,686][51532] Rollout worker 10 uses device cpu -[2023-10-15 14:47:47,686][51532] Rollout worker 11 uses device cpu -[2023-10-15 14:47:47,687][51532] Rollout worker 12 uses device cpu -[2023-10-15 14:47:47,687][51532] Rollout worker 13 uses device cpu -[2023-10-15 14:47:47,687][51532] Rollout worker 14 uses device cpu -[2023-10-15 14:47:47,688][51532] Rollout worker 15 uses device cpu -[2023-10-15 14:47:47,987][51532] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-15 14:47:47,987][51532] InferenceWorker_p0-w0: min num requests: 2 -[2023-10-15 14:47:47,991][51532] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-10-15 14:47:47,991][51532] InferenceWorker_p1-w0: min num requests: 2 -[2023-10-15 14:47:48,038][51532] Starting all processes... -[2023-10-15 14:47:48,038][51532] Starting process learner_proc0 -[2023-10-15 14:47:49,796][51532] Starting process learner_proc1 -[2023-10-15 14:47:49,799][52410] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-15 14:47:49,800][52410] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2023-10-15 14:47:49,818][52410] Num visible devices: 1 -[2023-10-15 14:47:49,835][52410] Setting fixed seed 1234 -[2023-10-15 14:47:49,837][52410] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-15 14:47:49,837][52410] Initializing actor-critic model on device cuda:0 -[2023-10-15 14:47:49,838][52410] RunningMeanStd input shape: (4, 84, 84) -[2023-10-15 14:47:49,838][52410] RunningMeanStd input shape: (1,) -[2023-10-15 14:47:49,856][52410] ConvEncoder: input_channels=4 -[2023-10-15 14:47:50,018][52410] Conv encoder output size: 512 -[2023-10-15 14:47:50,020][52410] Created Actor Critic model with architecture: -[2023-10-15 14:47:50,020][52410] ActorCriticSharedWeights( - (obs_normalizer): ObservationNormalizer( - (running_mean_std): RunningMeanStdDictInPlace( - (running_mean_std): ModuleDict( - (obs): RunningMeanStdInPlace() - ) - ) - ) - (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) - (encoder): MultiInputEncoder( - (encoders): ModuleDict( - (obs): ConvEncoder( - (enc): RecursiveScriptModule( - original_name=ConvEncoderImpl - (conv_head): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Conv2d) - (1): RecursiveScriptModule(original_name=ReLU) - (2): RecursiveScriptModule(original_name=Conv2d) - (3): RecursiveScriptModule(original_name=ReLU) - (4): RecursiveScriptModule(original_name=Conv2d) - (5): RecursiveScriptModule(original_name=ReLU) - ) - (mlp_layers): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Linear) - (1): RecursiveScriptModule(original_name=ReLU) - ) - ) - ) - ) - ) - (core): ModelCoreIdentity() - (decoder): MlpDecoder( - (mlp): Identity() - ) - (critic_linear): Linear(in_features=512, out_features=1, bias=True) - (action_parameterization): ActionParameterizationDefault( - (distribution_linear): Linear(in_features=512, out_features=6, bias=True) - ) -) -[2023-10-15 14:47:50,589][52410] Using optimizer -[2023-10-15 14:47:50,589][52410] No checkpoints found -[2023-10-15 14:47:50,590][52410] Did not load from checkpoint, starting from scratch! -[2023-10-15 14:47:50,590][52410] Initialized policy 0 weights for model version 0 -[2023-10-15 14:47:50,591][52410] LearnerWorker_p0 finished initialization! -[2023-10-15 14:47:50,591][52410] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-15 14:47:51,583][51532] Starting all processes... -[2023-10-15 14:47:51,586][52518] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-10-15 14:47:51,587][52518] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 -[2023-10-15 14:47:51,592][51532] Starting process inference_proc0-0 -[2023-10-15 14:47:51,592][51532] Starting process inference_proc1-0 -[2023-10-15 14:47:51,593][51532] Starting process rollout_proc0 -[2023-10-15 14:47:51,605][52518] Num visible devices: 1 -[2023-10-15 14:47:51,593][51532] Starting process rollout_proc1 -[2023-10-15 14:47:51,593][51532] Starting process rollout_proc2 -[2023-10-15 14:47:51,622][52518] Setting fixed seed 1234 -[2023-10-15 14:47:51,594][51532] Starting process rollout_proc3 -[2023-10-15 14:47:51,623][52518] Using GPUs [0] for process 1 (actually maps to GPUs [1]) -[2023-10-15 14:47:51,594][51532] Starting process rollout_proc4 -[2023-10-15 14:47:51,623][52518] Initializing actor-critic model on device cuda:0 -[2023-10-15 14:47:51,624][52518] RunningMeanStd input shape: (4, 84, 84) -[2023-10-15 14:47:51,624][52518] RunningMeanStd input shape: (1,) -[2023-10-15 14:47:51,597][51532] Starting process rollout_proc5 -[2023-10-15 14:47:51,598][51532] Starting process rollout_proc6 -[2023-10-15 14:47:51,598][51532] Starting process rollout_proc7 -[2023-10-15 14:47:51,599][51532] Starting process rollout_proc8 -[2023-10-15 14:47:51,599][51532] Starting process rollout_proc9 -[2023-10-15 14:47:51,637][52518] ConvEncoder: input_channels=4 -[2023-10-15 14:47:51,605][51532] Starting process rollout_proc10 -[2023-10-15 14:47:51,607][51532] Starting process rollout_proc11 -[2023-10-15 14:47:51,619][51532] Starting process rollout_proc12 -[2023-10-15 14:47:51,620][51532] Starting process rollout_proc13 -[2023-10-15 14:47:52,116][52518] Conv encoder output size: 512 -[2023-10-15 14:47:52,119][52518] Created Actor Critic model with architecture: -[2023-10-15 14:47:52,119][52518] ActorCriticSharedWeights( - (obs_normalizer): ObservationNormalizer( - (running_mean_std): RunningMeanStdDictInPlace( - (running_mean_std): ModuleDict( - (obs): RunningMeanStdInPlace() - ) - ) - ) - (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) - (encoder): MultiInputEncoder( - (encoders): ModuleDict( - (obs): ConvEncoder( - (enc): RecursiveScriptModule( - original_name=ConvEncoderImpl - (conv_head): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Conv2d) - (1): RecursiveScriptModule(original_name=ReLU) - (2): RecursiveScriptModule(original_name=Conv2d) - (3): RecursiveScriptModule(original_name=ReLU) - (4): RecursiveScriptModule(original_name=Conv2d) - (5): RecursiveScriptModule(original_name=ReLU) - ) - (mlp_layers): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Linear) - (1): RecursiveScriptModule(original_name=ReLU) - ) - ) - ) - ) - ) - (core): ModelCoreIdentity() - (decoder): MlpDecoder( - (mlp): Identity() - ) - (critic_linear): Linear(in_features=512, out_features=1, bias=True) - (action_parameterization): ActionParameterizationDefault( - (distribution_linear): Linear(in_features=512, out_features=6, bias=True) - ) -) -[2023-10-15 14:47:52,887][52518] Using optimizer -[2023-10-15 14:47:52,887][52518] No checkpoints found -[2023-10-15 14:47:52,887][52518] Did not load from checkpoint, starting from scratch! -[2023-10-15 14:47:52,888][52518] Initialized policy 1 weights for model version 0 -[2023-10-15 14:47:52,889][52518] LearnerWorker_p1 finished initialization! -[2023-10-15 14:47:52,889][52518] Using GPUs [0] for process 1 (actually maps to GPUs [1]) -[2023-10-15 14:47:53,832][51532] Starting process rollout_proc14 -[2023-10-15 14:47:53,840][52833] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-15 14:47:53,840][52833] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2023-10-15 14:47:53,862][52833] Num visible devices: 1 -[2023-10-15 14:47:53,881][52877] Worker 9 uses CPU cores [18, 19] -[2023-10-15 14:47:53,887][52878] Worker 10 uses CPU cores [20, 21] -[2023-10-15 14:47:53,890][51532] Starting process rollout_proc15 -[2023-10-15 14:47:53,901][52874] Worker 5 uses CPU cores [10, 11] -[2023-10-15 14:47:53,932][52879] Worker 7 uses CPU cores [14, 15] -[2023-10-15 14:47:53,955][52869] Worker 0 uses CPU cores [0, 1] -[2023-10-15 14:47:54,047][52872] Worker 4 uses CPU cores [8, 9] -[2023-10-15 14:47:54,108][52881] Worker 13 uses CPU cores [26, 27] -[2023-10-15 14:47:54,124][52866] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-10-15 14:47:54,124][52866] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 -[2023-10-15 14:47:54,149][52873] Worker 2 uses CPU cores [4, 5] -[2023-10-15 14:47:54,150][52866] Num visible devices: 1 -[2023-10-15 14:47:54,193][52875] Worker 6 uses CPU cores [12, 13] -[2023-10-15 14:47:54,233][52870] Worker 1 uses CPU cores [2, 3] -[2023-10-15 14:47:54,391][52882] Worker 11 uses CPU cores [22, 23] -[2023-10-15 14:47:54,391][52876] Worker 8 uses CPU cores [16, 17] -[2023-10-15 14:47:54,447][52880] Worker 12 uses CPU cores [24, 25] -[2023-10-15 14:47:54,530][52871] Worker 3 uses CPU cores [6, 7] -[2023-10-15 14:47:54,674][52833] RunningMeanStd input shape: (4, 84, 84) -[2023-10-15 14:47:54,675][52833] RunningMeanStd input shape: (1,) -[2023-10-15 14:47:54,686][52833] ConvEncoder: input_channels=4 -[2023-10-15 14:47:54,780][52866] RunningMeanStd input shape: (4, 84, 84) -[2023-10-15 14:47:54,781][52866] RunningMeanStd input shape: (1,) -[2023-10-15 14:47:54,790][52833] Conv encoder output size: 512 -[2023-10-15 14:47:54,792][52866] ConvEncoder: input_channels=4 -[2023-10-15 14:47:54,894][52866] Conv encoder output size: 512 -[2023-10-15 14:47:55,747][53658] Worker 15 uses CPU cores [30, 31] -[2023-10-15 14:47:55,802][51532] Inference worker 0-0 is ready! -[2023-10-15 14:47:55,803][53503] Worker 14 uses CPU cores [28, 29] -[2023-10-15 14:47:55,803][51532] Inference worker 1-0 is ready! -[2023-10-15 14:47:55,804][51532] All inference workers are ready! Signal rollout workers to start! -[2023-10-15 14:47:55,805][52881] EnvRunner 13-0 uses policy 1 -[2023-10-15 14:47:55,805][52871] EnvRunner 3-0 uses policy 1 -[2023-10-15 14:47:55,805][52879] EnvRunner 7-0 uses policy 1 -[2023-10-15 14:47:55,805][52873] EnvRunner 2-0 uses policy 0 -[2023-10-15 14:47:55,805][52882] EnvRunner 11-0 uses policy 1 -[2023-10-15 14:47:55,805][52875] EnvRunner 6-0 uses policy 0 -[2023-10-15 14:47:55,805][52870] EnvRunner 1-0 uses policy 1 -[2023-10-15 14:47:55,805][52880] EnvRunner 12-0 uses policy 0 -[2023-10-15 14:47:55,806][52872] EnvRunner 4-0 uses policy 0 -[2023-10-15 14:47:55,805][52869] EnvRunner 0-0 uses policy 0 -[2023-10-15 14:47:55,806][52878] EnvRunner 10-0 uses policy 0 -[2023-10-15 14:47:55,806][52877] EnvRunner 9-0 uses policy 1 -[2023-10-15 14:47:55,806][52876] EnvRunner 8-0 uses policy 0 -[2023-10-15 14:47:55,806][52874] EnvRunner 5-0 uses policy 1 -[2023-10-15 14:47:55,806][51532] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-10-15 14:47:55,918][53503] EnvRunner 14-0 uses policy 0 -[2023-10-15 14:47:55,930][53658] EnvRunner 15-0 uses policy 1 -[2023-10-15 14:47:57,974][51532] Heartbeat connected on Batcher_0 -[2023-10-15 14:47:57,977][51532] Heartbeat connected on LearnerWorker_p0 -[2023-10-15 14:47:57,980][51532] Heartbeat connected on Batcher_1 -[2023-10-15 14:47:57,983][51532] Heartbeat connected on LearnerWorker_p1 -[2023-10-15 14:47:57,990][51532] Heartbeat connected on InferenceWorker_p0-w0 -[2023-10-15 14:47:57,995][51532] Heartbeat connected on InferenceWorker_p1-w0 -[2023-10-15 14:47:57,999][51532] Heartbeat connected on RolloutWorker_w0 -[2023-10-15 14:47:58,000][51532] Heartbeat connected on RolloutWorker_w1 -[2023-10-15 14:47:58,003][51532] Heartbeat connected on RolloutWorker_w2 -[2023-10-15 14:47:58,004][51532] Heartbeat connected on RolloutWorker_w3 -[2023-10-15 14:47:58,008][51532] Heartbeat connected on RolloutWorker_w5 -[2023-10-15 14:47:58,010][51532] Heartbeat connected on RolloutWorker_w4 -[2023-10-15 14:47:58,014][51532] Heartbeat connected on RolloutWorker_w6 -[2023-10-15 14:47:58,017][51532] Heartbeat connected on RolloutWorker_w8 -[2023-10-15 14:47:58,019][51532] Heartbeat connected on RolloutWorker_w7 -[2023-10-15 14:47:58,020][51532] Heartbeat connected on RolloutWorker_w9 -[2023-10-15 14:47:58,028][51532] Heartbeat connected on RolloutWorker_w10 -[2023-10-15 14:47:58,029][51532] Heartbeat connected on RolloutWorker_w11 -[2023-10-15 14:47:58,031][51532] Heartbeat connected on RolloutWorker_w13 -[2023-10-15 14:47:58,034][51532] Heartbeat connected on RolloutWorker_w12 -[2023-10-15 14:47:58,039][51532] Heartbeat connected on RolloutWorker_w14 -[2023-10-15 14:47:58,041][51532] Heartbeat connected on RolloutWorker_w15 -[2023-10-15 14:47:58,441][51532] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 421.2, 1: 493.3. Samples: 2410. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-10-15 14:47:58,442][51532] Avg episode reward: [(0, '4.000'), (1, '3.250')] -[2023-10-15 14:48:03,441][51532] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 910.5, 1: 967.6. Samples: 14340. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-10-15 14:48:03,442][51532] Avg episode reward: [(0, '2.870'), (1, '2.902')] -[2023-10-15 14:48:05,827][52833] Updated weights for policy 0, policy_version 10 (0.0008) -[2023-10-15 14:48:05,828][52866] Updated weights for policy 1, policy_version 10 (0.0010) -[2023-10-15 14:48:06,181][52833] Updated weights for policy 0, policy_version 20 (0.0008) -[2023-10-15 14:48:06,195][52866] Updated weights for policy 1, policy_version 20 (0.0008) -[2023-10-15 14:48:06,552][52866] Updated weights for policy 1, policy_version 30 (0.0007) -[2023-10-15 14:48:06,555][52833] Updated weights for policy 0, policy_version 30 (0.0008) -[2023-10-15 14:48:08,441][51532] Fps is (10 sec: 6553.6, 60 sec: 5186.7, 300 sec: 5186.7). Total num frames: 65536. Throughput: 0: 1218.3, 1: 1241.9. Samples: 31086. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 14:48:08,442][51532] Avg episode reward: [(0, '2.950'), (1, '2.790')] -[2023-10-15 14:48:08,980][52866] Updated weights for policy 1, policy_version 40 (0.0009) -[2023-10-15 14:48:09,011][52833] Updated weights for policy 0, policy_version 40 (0.0010) -[2023-10-15 14:48:09,336][52866] Updated weights for policy 1, policy_version 50 (0.0007) -[2023-10-15 14:48:09,377][52833] Updated weights for policy 0, policy_version 50 (0.0009) -[2023-10-15 14:48:09,693][52866] Updated weights for policy 1, policy_version 60 (0.0009) -[2023-10-15 14:48:09,748][52833] Updated weights for policy 0, policy_version 60 (0.0008) -[2023-10-15 14:48:12,924][52866] Updated weights for policy 1, policy_version 70 (0.0007) -[2023-10-15 14:48:13,288][52866] Updated weights for policy 1, policy_version 80 (0.0008) -[2023-10-15 14:48:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 7432.3, 300 sec: 7432.3). Total num frames: 131072. Throughput: 0: 1464.8, 1: 1478.1. Samples: 51898. Policy #0 lag: (min: 33.0, avg: 33.0, max: 33.0) -[2023-10-15 14:48:13,442][51532] Avg episode reward: [(0, '2.870'), (1, '3.040')] -[2023-10-15 14:48:13,463][52833] Updated weights for policy 0, policy_version 70 (0.0007) -[2023-10-15 14:48:13,650][52866] Updated weights for policy 1, policy_version 90 (0.0008) -[2023-10-15 14:48:13,824][52833] Updated weights for policy 0, policy_version 80 (0.0008) -[2023-10-15 14:48:14,194][52833] Updated weights for policy 0, policy_version 90 (0.0007) -[2023-10-15 14:48:17,488][52866] Updated weights for policy 1, policy_version 100 (0.0009) -[2023-10-15 14:48:17,640][52833] Updated weights for policy 0, policy_version 100 (0.0010) -[2023-10-15 14:48:17,846][52866] Updated weights for policy 1, policy_version 110 (0.0008) -[2023-10-15 14:48:18,012][52833] Updated weights for policy 0, policy_version 110 (0.0008) -[2023-10-15 14:48:18,211][52866] Updated weights for policy 1, policy_version 120 (0.0007) -[2023-10-15 14:48:18,378][52833] Updated weights for policy 0, policy_version 120 (0.0007) -[2023-10-15 14:48:18,441][51532] Fps is (10 sec: 13106.8, 60 sec: 8685.7, 300 sec: 8685.7). Total num frames: 196608. Throughput: 0: 1349.0, 1: 1376.9. Samples: 61704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:48:18,442][51532] Avg episode reward: [(0, '2.960'), (1, '3.220')] -[2023-10-15 14:48:22,098][52866] Updated weights for policy 1, policy_version 130 (0.0008) -[2023-10-15 14:48:22,340][52833] Updated weights for policy 0, policy_version 130 (0.0008) -[2023-10-15 14:48:22,452][52866] Updated weights for policy 1, policy_version 140 (0.0008) -[2023-10-15 14:48:22,709][52833] Updated weights for policy 0, policy_version 140 (0.0007) -[2023-10-15 14:48:22,813][52866] Updated weights for policy 1, policy_version 150 (0.0008) -[2023-10-15 14:48:23,083][52833] Updated weights for policy 0, policy_version 150 (0.0007) -[2023-10-15 14:48:23,176][52866] Updated weights for policy 1, policy_version 160 (0.0007) -[2023-10-15 14:48:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 10671.6, 300 sec: 10671.6). Total num frames: 294912. Throughput: 0: 1494.8, 1: 1511.7. Samples: 83084. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) -[2023-10-15 14:48:23,441][51532] Avg episode reward: [(0, '3.450'), (1, '3.100')] -[2023-10-15 14:48:23,442][52518] Saving new best policy, reward=3.100! -[2023-10-15 14:48:23,444][52410] Saving new best policy, reward=3.450! -[2023-10-15 14:48:23,445][52833] Updated weights for policy 0, policy_version 160 (0.0008) -[2023-10-15 14:48:27,219][52866] Updated weights for policy 1, policy_version 170 (0.0007) -[2023-10-15 14:48:27,232][52833] Updated weights for policy 0, policy_version 170 (0.0008) -[2023-10-15 14:48:27,580][52866] Updated weights for policy 1, policy_version 180 (0.0009) -[2023-10-15 14:48:27,602][52833] Updated weights for policy 0, policy_version 180 (0.0009) -[2023-10-15 14:48:27,946][52866] Updated weights for policy 1, policy_version 190 (0.0008) -[2023-10-15 14:48:27,967][52833] Updated weights for policy 0, policy_version 190 (0.0007) -[2023-10-15 14:48:28,441][51532] Fps is (10 sec: 19661.4, 60 sec: 12048.8, 300 sec: 12048.8). Total num frames: 393216. Throughput: 0: 1564.1, 1: 1565.9. Samples: 102148. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) -[2023-10-15 14:48:28,442][51532] Avg episode reward: [(0, '3.300'), (1, '3.230')] -[2023-10-15 14:48:28,452][52518] Saving new best policy, reward=3.230! -[2023-10-15 14:48:31,909][52866] Updated weights for policy 1, policy_version 200 (0.0008) -[2023-10-15 14:48:31,938][52833] Updated weights for policy 0, policy_version 200 (0.0008) -[2023-10-15 14:48:32,278][52866] Updated weights for policy 1, policy_version 210 (0.0008) -[2023-10-15 14:48:32,297][52833] Updated weights for policy 0, policy_version 210 (0.0007) -[2023-10-15 14:48:32,639][52866] Updated weights for policy 1, policy_version 220 (0.0009) -[2023-10-15 14:48:32,663][52833] Updated weights for policy 0, policy_version 220 (0.0007) -[2023-10-15 14:48:33,441][51532] Fps is (10 sec: 16383.7, 60 sec: 12189.4, 300 sec: 12189.4). Total num frames: 458752. Throughput: 0: 1510.0, 1: 1514.9. Samples: 113840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:48:33,442][51532] Avg episode reward: [(0, '3.460'), (1, '3.460')] -[2023-10-15 14:48:33,443][52410] Saving new best policy, reward=3.460! -[2023-10-15 14:48:33,443][52518] Saving new best policy, reward=3.460! -[2023-10-15 14:48:36,633][52866] Updated weights for policy 1, policy_version 230 (0.0007) -[2023-10-15 14:48:36,686][52833] Updated weights for policy 0, policy_version 230 (0.0007) -[2023-10-15 14:48:36,991][52866] Updated weights for policy 1, policy_version 240 (0.0007) -[2023-10-15 14:48:37,050][52833] Updated weights for policy 0, policy_version 240 (0.0008) -[2023-10-15 14:48:37,352][52866] Updated weights for policy 1, policy_version 250 (0.0008) -[2023-10-15 14:48:37,416][52833] Updated weights for policy 0, policy_version 250 (0.0007) -[2023-10-15 14:48:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 12297.0, 300 sec: 12297.0). Total num frames: 524288. Throughput: 0: 1571.7, 1: 1568.4. Samples: 133876. Policy #0 lag: (min: 4.0, avg: 10.5, max: 36.0) -[2023-10-15 14:48:38,442][51532] Avg episode reward: [(0, '3.560'), (1, '3.240')] -[2023-10-15 14:48:38,443][52410] Saving new best policy, reward=3.560! -[2023-10-15 14:48:41,249][52866] Updated weights for policy 1, policy_version 260 (0.0008) -[2023-10-15 14:48:41,414][52833] Updated weights for policy 0, policy_version 260 (0.0008) -[2023-10-15 14:48:41,606][52866] Updated weights for policy 1, policy_version 270 (0.0008) -[2023-10-15 14:48:41,779][52833] Updated weights for policy 0, policy_version 270 (0.0007) -[2023-10-15 14:48:41,976][52866] Updated weights for policy 1, policy_version 280 (0.0007) -[2023-10-15 14:48:42,152][52833] Updated weights for policy 0, policy_version 280 (0.0007) -[2023-10-15 14:48:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 12382.1, 300 sec: 12382.1). Total num frames: 589824. Throughput: 0: 1678.7, 1: 1682.9. Samples: 153684. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-15 14:48:43,442][51532] Avg episode reward: [(0, '3.530'), (1, '3.060')] -[2023-10-15 14:48:46,026][52866] Updated weights for policy 1, policy_version 290 (0.0008) -[2023-10-15 14:48:46,049][52833] Updated weights for policy 0, policy_version 290 (0.0009) -[2023-10-15 14:48:46,396][52866] Updated weights for policy 1, policy_version 300 (0.0008) -[2023-10-15 14:48:46,424][52833] Updated weights for policy 0, policy_version 300 (0.0007) -[2023-10-15 14:48:46,762][52866] Updated weights for policy 1, policy_version 310 (0.0009) -[2023-10-15 14:48:46,792][52833] Updated weights for policy 0, policy_version 310 (0.0008) -[2023-10-15 14:48:47,123][52866] Updated weights for policy 1, policy_version 320 (0.0008) -[2023-10-15 14:48:47,164][52833] Updated weights for policy 0, policy_version 320 (0.0007) -[2023-10-15 14:48:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 12451.0, 300 sec: 12451.0). Total num frames: 655360. Throughput: 0: 1681.4, 1: 1680.8. Samples: 165636. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) -[2023-10-15 14:48:48,442][51532] Avg episode reward: [(0, '3.740'), (1, '3.130')] -[2023-10-15 14:48:48,442][52410] Saving new best policy, reward=3.740! -[2023-10-15 14:48:51,074][52866] Updated weights for policy 1, policy_version 330 (0.0007) -[2023-10-15 14:48:51,076][52833] Updated weights for policy 0, policy_version 330 (0.0009) -[2023-10-15 14:48:51,430][52866] Updated weights for policy 1, policy_version 340 (0.0008) -[2023-10-15 14:48:51,445][52833] Updated weights for policy 0, policy_version 340 (0.0008) -[2023-10-15 14:48:51,797][52866] Updated weights for policy 1, policy_version 350 (0.0007) -[2023-10-15 14:48:51,818][52833] Updated weights for policy 0, policy_version 350 (0.0008) -[2023-10-15 14:48:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 12507.9, 300 sec: 12507.9). Total num frames: 720896. Throughput: 0: 1703.8, 1: 1704.9. Samples: 184480. Policy #0 lag: (min: 26.0, avg: 26.0, max: 29.0) -[2023-10-15 14:48:53,441][51532] Avg episode reward: [(0, '3.660'), (1, '3.730')] -[2023-10-15 14:48:53,442][52518] Saving new best policy, reward=3.730! -[2023-10-15 14:48:55,748][52866] Updated weights for policy 1, policy_version 360 (0.0007) -[2023-10-15 14:48:55,900][52833] Updated weights for policy 0, policy_version 360 (0.0008) -[2023-10-15 14:48:56,109][52866] Updated weights for policy 1, policy_version 370 (0.0007) -[2023-10-15 14:48:56,278][52833] Updated weights for policy 0, policy_version 370 (0.0007) -[2023-10-15 14:48:56,471][52866] Updated weights for policy 1, policy_version 380 (0.0008) -[2023-10-15 14:48:56,639][52833] Updated weights for policy 0, policy_version 380 (0.0008) -[2023-10-15 14:48:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12555.7). Total num frames: 786432. Throughput: 0: 1705.7, 1: 1711.0. Samples: 205650. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 14:48:58,442][51532] Avg episode reward: [(0, '3.410'), (1, '3.670')] -[2023-10-15 14:49:00,376][52866] Updated weights for policy 1, policy_version 390 (0.0009) -[2023-10-15 14:49:00,532][52833] Updated weights for policy 0, policy_version 390 (0.0009) -[2023-10-15 14:49:00,733][52866] Updated weights for policy 1, policy_version 400 (0.0009) -[2023-10-15 14:49:00,903][52833] Updated weights for policy 0, policy_version 400 (0.0007) -[2023-10-15 14:49:01,098][52866] Updated weights for policy 1, policy_version 410 (0.0008) -[2023-10-15 14:49:01,277][52833] Updated weights for policy 0, policy_version 410 (0.0009) -[2023-10-15 14:49:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 12596.5). Total num frames: 851968. Throughput: 0: 1723.0, 1: 1712.2. Samples: 216286. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 14:49:03,442][51532] Avg episode reward: [(0, '3.710'), (1, '3.980')] -[2023-10-15 14:49:03,443][52518] Saving new best policy, reward=3.980! -[2023-10-15 14:49:05,096][52866] Updated weights for policy 1, policy_version 420 (0.0008) -[2023-10-15 14:49:05,355][52833] Updated weights for policy 0, policy_version 420 (0.0007) -[2023-10-15 14:49:05,452][52866] Updated weights for policy 1, policy_version 430 (0.0007) -[2023-10-15 14:49:05,733][52833] Updated weights for policy 0, policy_version 430 (0.0007) -[2023-10-15 14:49:05,820][52866] Updated weights for policy 1, policy_version 440 (0.0007) -[2023-10-15 14:49:06,106][52833] Updated weights for policy 0, policy_version 440 (0.0008) -[2023-10-15 14:49:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 12631.7). Total num frames: 917504. Throughput: 0: 1702.5, 1: 1702.7. Samples: 236316. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:49:08,442][51532] Avg episode reward: [(0, '3.690'), (1, '4.160')] -[2023-10-15 14:49:08,442][52518] Saving new best policy, reward=4.160! -[2023-10-15 14:49:09,691][52866] Updated weights for policy 1, policy_version 450 (0.0010) -[2023-10-15 14:49:09,773][52833] Updated weights for policy 0, policy_version 450 (0.0009) -[2023-10-15 14:49:10,047][52866] Updated weights for policy 1, policy_version 460 (0.0007) -[2023-10-15 14:49:10,141][52833] Updated weights for policy 0, policy_version 460 (0.0007) -[2023-10-15 14:49:10,408][52866] Updated weights for policy 1, policy_version 470 (0.0008) -[2023-10-15 14:49:10,512][52833] Updated weights for policy 0, policy_version 470 (0.0008) -[2023-10-15 14:49:10,766][52866] Updated weights for policy 1, policy_version 480 (0.0009) -[2023-10-15 14:49:10,883][52833] Updated weights for policy 0, policy_version 480 (0.0009) -[2023-10-15 14:49:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 12662.3). Total num frames: 983040. Throughput: 0: 1725.5, 1: 1739.9. Samples: 258090. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) -[2023-10-15 14:49:13,442][51532] Avg episode reward: [(0, '3.530'), (1, '3.810')] -[2023-10-15 14:49:14,642][52866] Updated weights for policy 1, policy_version 490 (0.0010) -[2023-10-15 14:49:14,999][52866] Updated weights for policy 1, policy_version 500 (0.0008) -[2023-10-15 14:49:15,014][52833] Updated weights for policy 0, policy_version 490 (0.0008) -[2023-10-15 14:49:15,361][52866] Updated weights for policy 1, policy_version 510 (0.0008) -[2023-10-15 14:49:15,382][52833] Updated weights for policy 0, policy_version 500 (0.0009) -[2023-10-15 14:49:15,767][52833] Updated weights for policy 0, policy_version 510 (0.0009) -[2023-10-15 14:49:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 12689.2). Total num frames: 1048576. Throughput: 0: 1700.5, 1: 1713.4. Samples: 267466. Policy #0 lag: (min: 26.0, avg: 34.5, max: 58.0) -[2023-10-15 14:49:18,441][51532] Avg episode reward: [(0, '4.240'), (1, '3.390')] -[2023-10-15 14:49:18,442][52410] Saving new best policy, reward=4.240! -[2023-10-15 14:49:19,209][52866] Updated weights for policy 1, policy_version 520 (0.0008) -[2023-10-15 14:49:19,571][52866] Updated weights for policy 1, policy_version 530 (0.0008) -[2023-10-15 14:49:19,629][52833] Updated weights for policy 0, policy_version 520 (0.0009) -[2023-10-15 14:49:19,937][52866] Updated weights for policy 1, policy_version 540 (0.0009) -[2023-10-15 14:49:20,001][52833] Updated weights for policy 0, policy_version 530 (0.0010) -[2023-10-15 14:49:20,374][52833] Updated weights for policy 0, policy_version 540 (0.0011) -[2023-10-15 14:49:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 12713.0). Total num frames: 1114112. Throughput: 0: 1709.0, 1: 1734.7. Samples: 288842. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) -[2023-10-15 14:49:23,442][51532] Avg episode reward: [(0, '4.540'), (1, '3.440')] -[2023-10-15 14:49:23,443][52410] Saving new best policy, reward=4.540! -[2023-10-15 14:49:23,936][52866] Updated weights for policy 1, policy_version 550 (0.0011) -[2023-10-15 14:49:24,244][52833] Updated weights for policy 0, policy_version 550 (0.0008) -[2023-10-15 14:49:24,297][52866] Updated weights for policy 1, policy_version 560 (0.0007) -[2023-10-15 14:49:24,618][52833] Updated weights for policy 0, policy_version 560 (0.0007) -[2023-10-15 14:49:24,665][52866] Updated weights for policy 1, policy_version 570 (0.0007) -[2023-10-15 14:49:24,979][52833] Updated weights for policy 0, policy_version 570 (0.0008) -[2023-10-15 14:49:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12734.3). Total num frames: 1179648. Throughput: 0: 1729.3, 1: 1751.0. Samples: 310296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:49:28,442][51532] Avg episode reward: [(0, '3.890'), (1, '3.600')] -[2023-10-15 14:49:28,518][52866] Updated weights for policy 1, policy_version 580 (0.0007) -[2023-10-15 14:49:28,877][52866] Updated weights for policy 1, policy_version 590 (0.0009) -[2023-10-15 14:49:29,017][52833] Updated weights for policy 0, policy_version 580 (0.0008) -[2023-10-15 14:49:29,238][52866] Updated weights for policy 1, policy_version 600 (0.0009) -[2023-10-15 14:49:29,375][52833] Updated weights for policy 0, policy_version 590 (0.0007) -[2023-10-15 14:49:29,744][52833] Updated weights for policy 0, policy_version 600 (0.0010) -[2023-10-15 14:49:33,070][52866] Updated weights for policy 1, policy_version 610 (0.0008) -[2023-10-15 14:49:33,431][52866] Updated weights for policy 1, policy_version 620 (0.0007) -[2023-10-15 14:49:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12753.4). Total num frames: 1245184. Throughput: 0: 1700.2, 1: 1729.1. Samples: 319952. Policy #0 lag: (min: 17.0, avg: 23.3, max: 49.0) -[2023-10-15 14:49:33,442][51532] Avg episode reward: [(0, '4.660'), (1, '3.710')] -[2023-10-15 14:49:33,443][52410] Saving new best policy, reward=4.660! -[2023-10-15 14:49:33,709][52833] Updated weights for policy 0, policy_version 610 (0.0010) -[2023-10-15 14:49:33,795][52866] Updated weights for policy 1, policy_version 630 (0.0009) -[2023-10-15 14:49:34,078][52833] Updated weights for policy 0, policy_version 620 (0.0007) -[2023-10-15 14:49:34,164][52866] Updated weights for policy 1, policy_version 640 (0.0008) -[2023-10-15 14:49:34,442][52833] Updated weights for policy 0, policy_version 630 (0.0007) -[2023-10-15 14:49:34,813][52833] Updated weights for policy 0, policy_version 640 (0.0009) -[2023-10-15 14:49:38,167][52866] Updated weights for policy 1, policy_version 650 (0.0007) -[2023-10-15 14:49:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12770.7). Total num frames: 1310720. Throughput: 0: 1728.4, 1: 1755.8. Samples: 341266. Policy #0 lag: (min: 28.0, avg: 38.5, max: 60.0) -[2023-10-15 14:49:38,441][51532] Avg episode reward: [(0, '5.020'), (1, '4.060')] -[2023-10-15 14:49:38,529][52866] Updated weights for policy 1, policy_version 660 (0.0009) -[2023-10-15 14:49:38,719][52833] Updated weights for policy 0, policy_version 650 (0.0008) -[2023-10-15 14:49:38,899][52866] Updated weights for policy 1, policy_version 670 (0.0010) -[2023-10-15 14:49:39,085][52833] Updated weights for policy 0, policy_version 660 (0.0008) -[2023-10-15 14:49:39,462][52833] Updated weights for policy 0, policy_version 670 (0.0009) -[2023-10-15 14:49:39,537][52410] Saving new best policy, reward=5.020! -[2023-10-15 14:49:42,734][52866] Updated weights for policy 1, policy_version 680 (0.0008) -[2023-10-15 14:49:43,110][52866] Updated weights for policy 1, policy_version 690 (0.0007) -[2023-10-15 14:49:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12786.3). Total num frames: 1376256. Throughput: 0: 1742.1, 1: 1749.0. Samples: 362750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:49:43,441][51532] Avg episode reward: [(0, '4.630'), (1, '4.090')] -[2023-10-15 14:49:43,445][52833] Updated weights for policy 0, policy_version 680 (0.0007) -[2023-10-15 14:49:43,476][52866] Updated weights for policy 1, policy_version 700 (0.0009) -[2023-10-15 14:49:43,621][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000000704_720896.pth... -[2023-10-15 14:49:43,835][52833] Updated weights for policy 0, policy_version 690 (0.0008) -[2023-10-15 14:49:44,208][52833] Updated weights for policy 0, policy_version 700 (0.0008) -[2023-10-15 14:49:44,350][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000000704_720896.pth... -[2023-10-15 14:49:47,328][52866] Updated weights for policy 1, policy_version 710 (0.0007) -[2023-10-15 14:49:47,685][52866] Updated weights for policy 1, policy_version 720 (0.0008) -[2023-10-15 14:49:48,054][52866] Updated weights for policy 1, policy_version 730 (0.0009) -[2023-10-15 14:49:48,076][52833] Updated weights for policy 0, policy_version 710 (0.0007) -[2023-10-15 14:49:48,441][51532] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13091.5). Total num frames: 1474560. Throughput: 0: 1720.5, 1: 1753.2. Samples: 372604. Policy #0 lag: (min: 15.0, avg: 24.0, max: 47.0) -[2023-10-15 14:49:48,441][51532] Avg episode reward: [(0, '4.410'), (1, '4.140')] -[2023-10-15 14:49:48,449][52833] Updated weights for policy 0, policy_version 720 (0.0009) -[2023-10-15 14:49:48,809][52833] Updated weights for policy 0, policy_version 730 (0.0009) -[2023-10-15 14:49:52,107][52866] Updated weights for policy 1, policy_version 740 (0.0010) -[2023-10-15 14:49:52,486][52866] Updated weights for policy 1, policy_version 750 (0.0008) -[2023-10-15 14:49:52,714][52833] Updated weights for policy 0, policy_version 740 (0.0010) -[2023-10-15 14:49:52,850][52866] Updated weights for policy 1, policy_version 760 (0.0007) -[2023-10-15 14:49:53,079][52833] Updated weights for policy 0, policy_version 750 (0.0008) -[2023-10-15 14:49:53,441][51532] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13092.1). Total num frames: 1540096. Throughput: 0: 1739.2, 1: 1763.3. Samples: 393928. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-15 14:49:53,441][51532] Avg episode reward: [(0, '4.980'), (1, '4.450')] -[2023-10-15 14:49:53,442][52518] Saving new best policy, reward=4.450! -[2023-10-15 14:49:53,449][52833] Updated weights for policy 0, policy_version 760 (0.0008) -[2023-10-15 14:49:56,938][52866] Updated weights for policy 1, policy_version 770 (0.0008) -[2023-10-15 14:49:57,303][52833] Updated weights for policy 0, policy_version 770 (0.0008) -[2023-10-15 14:49:57,309][52866] Updated weights for policy 1, policy_version 780 (0.0010) -[2023-10-15 14:49:57,669][52866] Updated weights for policy 1, policy_version 790 (0.0008) -[2023-10-15 14:49:57,678][52833] Updated weights for policy 0, policy_version 780 (0.0007) -[2023-10-15 14:49:58,035][52866] Updated weights for policy 1, policy_version 800 (0.0007) -[2023-10-15 14:49:58,047][52833] Updated weights for policy 0, policy_version 790 (0.0007) -[2023-10-15 14:49:58,412][52833] Updated weights for policy 0, policy_version 800 (0.0009) -[2023-10-15 14:49:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13360.0). Total num frames: 1638400. Throughput: 0: 1729.4, 1: 1729.0. Samples: 413718. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 14:49:58,441][51532] Avg episode reward: [(0, '4.980'), (1, '4.280')] -[2023-10-15 14:50:01,841][52866] Updated weights for policy 1, policy_version 810 (0.0007) -[2023-10-15 14:50:02,208][52866] Updated weights for policy 1, policy_version 820 (0.0008) -[2023-10-15 14:50:02,325][52833] Updated weights for policy 0, policy_version 810 (0.0007) -[2023-10-15 14:50:02,582][52866] Updated weights for policy 1, policy_version 830 (0.0008) -[2023-10-15 14:50:02,692][52833] Updated weights for policy 0, policy_version 820 (0.0007) -[2023-10-15 14:50:03,063][52833] Updated weights for policy 0, policy_version 830 (0.0008) -[2023-10-15 14:50:03,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13350.0). Total num frames: 1703936. Throughput: 0: 1747.5, 1: 1753.9. Samples: 425030. Policy #0 lag: (min: 12.0, avg: 17.3, max: 44.0) -[2023-10-15 14:50:03,442][51532] Avg episode reward: [(0, '5.340'), (1, '4.040')] -[2023-10-15 14:50:03,443][52410] Saving new best policy, reward=5.340! -[2023-10-15 14:50:06,466][52866] Updated weights for policy 1, policy_version 840 (0.0008) -[2023-10-15 14:50:06,835][52866] Updated weights for policy 1, policy_version 850 (0.0008) -[2023-10-15 14:50:06,840][52833] Updated weights for policy 0, policy_version 840 (0.0008) -[2023-10-15 14:50:07,200][52866] Updated weights for policy 1, policy_version 860 (0.0007) -[2023-10-15 14:50:07,220][52833] Updated weights for policy 0, policy_version 850 (0.0008) -[2023-10-15 14:50:07,582][52833] Updated weights for policy 0, policy_version 860 (0.0010) -[2023-10-15 14:50:08,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 13340.9). Total num frames: 1769472. Throughput: 0: 1743.8, 1: 1728.4. Samples: 445092. Policy #0 lag: (min: 8.0, avg: 34.9, max: 40.0) -[2023-10-15 14:50:08,442][51532] Avg episode reward: [(0, '5.200'), (1, '4.060')] -[2023-10-15 14:50:11,121][52866] Updated weights for policy 1, policy_version 870 (0.0007) -[2023-10-15 14:50:11,487][52866] Updated weights for policy 1, policy_version 880 (0.0010) -[2023-10-15 14:50:11,628][52833] Updated weights for policy 0, policy_version 870 (0.0007) -[2023-10-15 14:50:11,853][52866] Updated weights for policy 1, policy_version 890 (0.0008) -[2023-10-15 14:50:11,995][52833] Updated weights for policy 0, policy_version 880 (0.0007) -[2023-10-15 14:50:12,372][52833] Updated weights for policy 0, policy_version 890 (0.0007) -[2023-10-15 14:50:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13332.4). Total num frames: 1835008. Throughput: 0: 1723.5, 1: 1720.0. Samples: 465252. Policy #0 lag: (min: 16.0, avg: 36.5, max: 48.0) -[2023-10-15 14:50:13,442][51532] Avg episode reward: [(0, '5.150'), (1, '4.100')] -[2023-10-15 14:50:15,791][52866] Updated weights for policy 1, policy_version 900 (0.0009) -[2023-10-15 14:50:16,161][52866] Updated weights for policy 1, policy_version 910 (0.0009) -[2023-10-15 14:50:16,418][52833] Updated weights for policy 0, policy_version 900 (0.0007) -[2023-10-15 14:50:16,530][52866] Updated weights for policy 1, policy_version 920 (0.0008) -[2023-10-15 14:50:16,789][52833] Updated weights for policy 0, policy_version 910 (0.0008) -[2023-10-15 14:50:17,164][52833] Updated weights for policy 0, policy_version 920 (0.0009) -[2023-10-15 14:50:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13324.5). Total num frames: 1900544. Throughput: 0: 1751.7, 1: 1736.6. Samples: 476926. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 14:50:18,442][51532] Avg episode reward: [(0, '5.310'), (1, '4.600')] -[2023-10-15 14:50:18,443][52518] Saving new best policy, reward=4.600! -[2023-10-15 14:50:20,466][52866] Updated weights for policy 1, policy_version 930 (0.0008) -[2023-10-15 14:50:20,830][52866] Updated weights for policy 1, policy_version 940 (0.0008) -[2023-10-15 14:50:21,064][52833] Updated weights for policy 0, policy_version 930 (0.0007) -[2023-10-15 14:50:21,206][52866] Updated weights for policy 1, policy_version 950 (0.0009) -[2023-10-15 14:50:21,434][52833] Updated weights for policy 0, policy_version 940 (0.0009) -[2023-10-15 14:50:21,564][52866] Updated weights for policy 1, policy_version 960 (0.0008) -[2023-10-15 14:50:21,810][52833] Updated weights for policy 0, policy_version 950 (0.0010) -[2023-10-15 14:50:22,192][52833] Updated weights for policy 0, policy_version 960 (0.0010) -[2023-10-15 14:50:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13317.1). Total num frames: 1966080. Throughput: 0: 1730.2, 1: 1719.0. Samples: 496478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:50:23,442][51532] Avg episode reward: [(0, '5.270'), (1, '4.820')] -[2023-10-15 14:50:23,443][52518] Saving new best policy, reward=4.820! -[2023-10-15 14:50:25,500][52866] Updated weights for policy 1, policy_version 970 (0.0007) -[2023-10-15 14:50:25,871][52866] Updated weights for policy 1, policy_version 980 (0.0009) -[2023-10-15 14:50:26,046][52833] Updated weights for policy 0, policy_version 970 (0.0008) -[2023-10-15 14:50:26,233][52866] Updated weights for policy 1, policy_version 990 (0.0007) -[2023-10-15 14:50:26,418][52833] Updated weights for policy 0, policy_version 980 (0.0009) -[2023-10-15 14:50:26,783][52833] Updated weights for policy 0, policy_version 990 (0.0010) -[2023-10-15 14:50:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13310.3). Total num frames: 2031616. Throughput: 0: 1717.8, 1: 1724.9. Samples: 517672. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 14:50:28,441][51532] Avg episode reward: [(0, '5.550'), (1, '5.170')] -[2023-10-15 14:50:28,453][52410] Saving new best policy, reward=5.550! -[2023-10-15 14:50:28,453][52518] Saving new best policy, reward=5.170! -[2023-10-15 14:50:30,115][52866] Updated weights for policy 1, policy_version 1000 (0.0009) -[2023-10-15 14:50:30,494][52866] Updated weights for policy 1, policy_version 1010 (0.0008) -[2023-10-15 14:50:30,808][52833] Updated weights for policy 0, policy_version 1000 (0.0008) -[2023-10-15 14:50:30,850][52866] Updated weights for policy 1, policy_version 1020 (0.0007) -[2023-10-15 14:50:31,183][52833] Updated weights for policy 0, policy_version 1010 (0.0009) -[2023-10-15 14:50:31,558][52833] Updated weights for policy 0, policy_version 1020 (0.0009) -[2023-10-15 14:50:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13303.8). Total num frames: 2097152. Throughput: 0: 1740.7, 1: 1713.9. Samples: 528060. Policy #0 lag: (min: 13.0, avg: 14.7, max: 40.0) -[2023-10-15 14:50:33,442][51532] Avg episode reward: [(0, '6.250'), (1, '5.080')] -[2023-10-15 14:50:33,443][52410] Saving new best policy, reward=6.250! -[2023-10-15 14:50:34,730][52866] Updated weights for policy 1, policy_version 1030 (0.0009) -[2023-10-15 14:50:35,101][52866] Updated weights for policy 1, policy_version 1040 (0.0008) -[2023-10-15 14:50:35,358][52833] Updated weights for policy 0, policy_version 1030 (0.0008) -[2023-10-15 14:50:35,469][52866] Updated weights for policy 1, policy_version 1050 (0.0008) -[2023-10-15 14:50:35,717][52833] Updated weights for policy 0, policy_version 1040 (0.0010) -[2023-10-15 14:50:36,087][52833] Updated weights for policy 0, policy_version 1050 (0.0008) -[2023-10-15 14:50:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13297.8). Total num frames: 2162688. Throughput: 0: 1722.3, 1: 1713.4. Samples: 548538. Policy #0 lag: (min: 3.0, avg: 13.8, max: 35.0) -[2023-10-15 14:50:38,442][51532] Avg episode reward: [(0, '6.680'), (1, '5.090')] -[2023-10-15 14:50:38,443][52410] Saving new best policy, reward=6.680! -[2023-10-15 14:50:39,515][52866] Updated weights for policy 1, policy_version 1060 (0.0010) -[2023-10-15 14:50:39,873][52866] Updated weights for policy 1, policy_version 1070 (0.0009) -[2023-10-15 14:50:39,996][52833] Updated weights for policy 0, policy_version 1060 (0.0010) -[2023-10-15 14:50:40,236][52866] Updated weights for policy 1, policy_version 1080 (0.0007) -[2023-10-15 14:50:40,357][52833] Updated weights for policy 0, policy_version 1070 (0.0008) -[2023-10-15 14:50:40,730][52833] Updated weights for policy 0, policy_version 1080 (0.0008) -[2023-10-15 14:50:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13292.1). Total num frames: 2228224. Throughput: 0: 1736.2, 1: 1742.3. Samples: 570248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:50:43,441][51532] Avg episode reward: [(0, '6.620'), (1, '5.130')] -[2023-10-15 14:50:44,082][52866] Updated weights for policy 1, policy_version 1090 (0.0007) -[2023-10-15 14:50:44,455][52866] Updated weights for policy 1, policy_version 1100 (0.0011) -[2023-10-15 14:50:44,477][52833] Updated weights for policy 0, policy_version 1090 (0.0008) -[2023-10-15 14:50:44,809][52866] Updated weights for policy 1, policy_version 1110 (0.0008) -[2023-10-15 14:50:44,847][52833] Updated weights for policy 0, policy_version 1100 (0.0007) -[2023-10-15 14:50:45,177][52866] Updated weights for policy 1, policy_version 1120 (0.0008) -[2023-10-15 14:50:45,213][52833] Updated weights for policy 0, policy_version 1110 (0.0008) -[2023-10-15 14:50:45,580][52833] Updated weights for policy 0, policy_version 1120 (0.0010) -[2023-10-15 14:50:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13286.7). Total num frames: 2293760. Throughput: 0: 1719.0, 1: 1716.3. Samples: 579618. Policy #0 lag: (min: 6.0, avg: 8.2, max: 38.0) -[2023-10-15 14:50:48,441][51532] Avg episode reward: [(0, '5.950'), (1, '5.670')] -[2023-10-15 14:50:48,442][52518] Saving new best policy, reward=5.670! -[2023-10-15 14:50:49,175][52866] Updated weights for policy 1, policy_version 1130 (0.0009) -[2023-10-15 14:50:49,536][52833] Updated weights for policy 0, policy_version 1130 (0.0008) -[2023-10-15 14:50:49,540][52866] Updated weights for policy 1, policy_version 1140 (0.0009) -[2023-10-15 14:50:49,897][52833] Updated weights for policy 0, policy_version 1140 (0.0009) -[2023-10-15 14:50:49,901][52866] Updated weights for policy 1, policy_version 1150 (0.0008) -[2023-10-15 14:50:50,270][52833] Updated weights for policy 0, policy_version 1150 (0.0009) -[2023-10-15 14:50:53,441][51532] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13281.7). Total num frames: 2359296. Throughput: 0: 1724.9, 1: 1734.5. Samples: 600768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:50:53,442][51532] Avg episode reward: [(0, '5.770'), (1, '5.820')] -[2023-10-15 14:50:53,768][52866] Updated weights for policy 1, policy_version 1160 (0.0007) -[2023-10-15 14:50:54,031][52833] Updated weights for policy 0, policy_version 1160 (0.0007) -[2023-10-15 14:50:54,128][52866] Updated weights for policy 1, policy_version 1170 (0.0008) -[2023-10-15 14:50:54,400][52833] Updated weights for policy 0, policy_version 1170 (0.0009) -[2023-10-15 14:50:54,495][52866] Updated weights for policy 1, policy_version 1180 (0.0007) -[2023-10-15 14:50:54,641][52518] Saving new best policy, reward=5.820! -[2023-10-15 14:50:54,772][52833] Updated weights for policy 0, policy_version 1180 (0.0009) -[2023-10-15 14:50:58,373][52866] Updated weights for policy 1, policy_version 1190 (0.0008) -[2023-10-15 14:50:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13276.9). Total num frames: 2424832. Throughput: 0: 1755.6, 1: 1745.6. Samples: 622808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:50:58,441][51532] Avg episode reward: [(0, '6.230'), (1, '5.540')] -[2023-10-15 14:50:58,582][52833] Updated weights for policy 0, policy_version 1190 (0.0008) -[2023-10-15 14:50:58,732][52866] Updated weights for policy 1, policy_version 1200 (0.0009) -[2023-10-15 14:50:58,958][52833] Updated weights for policy 0, policy_version 1200 (0.0008) -[2023-10-15 14:50:59,099][52866] Updated weights for policy 1, policy_version 1210 (0.0008) -[2023-10-15 14:50:59,325][52833] Updated weights for policy 0, policy_version 1210 (0.0008) -[2023-10-15 14:51:02,902][52866] Updated weights for policy 1, policy_version 1220 (0.0009) -[2023-10-15 14:51:03,281][52866] Updated weights for policy 1, policy_version 1230 (0.0010) -[2023-10-15 14:51:03,296][52833] Updated weights for policy 0, policy_version 1220 (0.0008) -[2023-10-15 14:51:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13272.4). Total num frames: 2490368. Throughput: 0: 1729.8, 1: 1725.4. Samples: 632410. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 14:51:03,442][51532] Avg episode reward: [(0, '6.580'), (1, '5.480')] -[2023-10-15 14:51:03,646][52866] Updated weights for policy 1, policy_version 1240 (0.0008) -[2023-10-15 14:51:03,658][52833] Updated weights for policy 0, policy_version 1230 (0.0008) -[2023-10-15 14:51:04,024][52833] Updated weights for policy 0, policy_version 1240 (0.0009) -[2023-10-15 14:51:07,436][52866] Updated weights for policy 1, policy_version 1250 (0.0008) -[2023-10-15 14:51:07,807][52866] Updated weights for policy 1, policy_version 1260 (0.0008) -[2023-10-15 14:51:07,951][52833] Updated weights for policy 0, policy_version 1250 (0.0009) -[2023-10-15 14:51:08,171][52866] Updated weights for policy 1, policy_version 1270 (0.0008) -[2023-10-15 14:51:08,312][52833] Updated weights for policy 0, policy_version 1260 (0.0007) -[2023-10-15 14:51:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13268.1). Total num frames: 2555904. Throughput: 0: 1746.7, 1: 1750.3. Samples: 653842. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) -[2023-10-15 14:51:08,442][51532] Avg episode reward: [(0, '6.140'), (1, '5.750')] -[2023-10-15 14:51:08,535][52866] Updated weights for policy 1, policy_version 1280 (0.0007) -[2023-10-15 14:51:08,685][52833] Updated weights for policy 0, policy_version 1270 (0.0007) -[2023-10-15 14:51:09,053][52833] Updated weights for policy 0, policy_version 1280 (0.0007) -[2023-10-15 14:51:12,440][52866] Updated weights for policy 1, policy_version 1290 (0.0009) -[2023-10-15 14:51:12,805][52866] Updated weights for policy 1, policy_version 1300 (0.0008) -[2023-10-15 14:51:13,050][52833] Updated weights for policy 0, policy_version 1290 (0.0008) -[2023-10-15 14:51:13,172][52866] Updated weights for policy 1, policy_version 1310 (0.0008) -[2023-10-15 14:51:13,428][52833] Updated weights for policy 0, policy_version 1300 (0.0010) -[2023-10-15 14:51:13,441][51532] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13429.8). Total num frames: 2654208. Throughput: 0: 1750.1, 1: 1732.9. Samples: 674410. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-15 14:51:13,441][51532] Avg episode reward: [(0, '6.220'), (1, '6.170')] -[2023-10-15 14:51:13,449][52518] Saving new best policy, reward=6.170! -[2023-10-15 14:51:13,793][52833] Updated weights for policy 0, policy_version 1310 (0.0009) -[2023-10-15 14:51:17,451][52866] Updated weights for policy 1, policy_version 1320 (0.0007) -[2023-10-15 14:51:17,823][52866] Updated weights for policy 1, policy_version 1330 (0.0007) -[2023-10-15 14:51:17,985][52833] Updated weights for policy 0, policy_version 1320 (0.0010) -[2023-10-15 14:51:18,186][52866] Updated weights for policy 1, policy_version 1340 (0.0008) -[2023-10-15 14:51:18,366][52833] Updated weights for policy 0, policy_version 1330 (0.0009) -[2023-10-15 14:51:18,441][51532] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13421.9). Total num frames: 2719744. Throughput: 0: 1732.3, 1: 1749.8. Samples: 684754. Policy #0 lag: (min: 26.0, avg: 26.0, max: 29.0) -[2023-10-15 14:51:18,441][51532] Avg episode reward: [(0, '6.420'), (1, '6.290')] -[2023-10-15 14:51:18,442][52518] Saving new best policy, reward=6.290! -[2023-10-15 14:51:18,743][52833] Updated weights for policy 0, policy_version 1340 (0.0009) -[2023-10-15 14:51:22,182][52866] Updated weights for policy 1, policy_version 1350 (0.0008) -[2023-10-15 14:51:22,542][52866] Updated weights for policy 1, policy_version 1360 (0.0007) -[2023-10-15 14:51:22,600][52833] Updated weights for policy 0, policy_version 1350 (0.0009) -[2023-10-15 14:51:22,912][52866] Updated weights for policy 1, policy_version 1370 (0.0009) -[2023-10-15 14:51:22,960][52833] Updated weights for policy 0, policy_version 1360 (0.0008) -[2023-10-15 14:51:23,331][52833] Updated weights for policy 0, policy_version 1370 (0.0008) -[2023-10-15 14:51:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13414.3). Total num frames: 2785280. Throughput: 0: 1755.9, 1: 1743.1. Samples: 705992. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) -[2023-10-15 14:51:23,441][51532] Avg episode reward: [(0, '6.490'), (1, '6.850')] -[2023-10-15 14:51:23,442][52518] Saving new best policy, reward=6.850! -[2023-10-15 14:51:26,856][52866] Updated weights for policy 1, policy_version 1380 (0.0008) -[2023-10-15 14:51:27,226][52866] Updated weights for policy 1, policy_version 1390 (0.0010) -[2023-10-15 14:51:27,272][52833] Updated weights for policy 0, policy_version 1380 (0.0007) -[2023-10-15 14:51:27,589][52866] Updated weights for policy 1, policy_version 1400 (0.0008) -[2023-10-15 14:51:27,644][52833] Updated weights for policy 0, policy_version 1390 (0.0009) -[2023-10-15 14:51:28,011][52833] Updated weights for policy 0, policy_version 1400 (0.0009) -[2023-10-15 14:51:28,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13561.2). Total num frames: 2883584. Throughput: 0: 1732.9, 1: 1712.2. Samples: 725276. Policy #0 lag: (min: 10.0, avg: 18.2, max: 42.0) -[2023-10-15 14:51:28,442][51532] Avg episode reward: [(0, '6.200'), (1, '6.600')] -[2023-10-15 14:51:31,482][52866] Updated weights for policy 1, policy_version 1410 (0.0008) -[2023-10-15 14:51:31,792][52833] Updated weights for policy 0, policy_version 1410 (0.0009) -[2023-10-15 14:51:31,852][52866] Updated weights for policy 1, policy_version 1420 (0.0008) -[2023-10-15 14:51:32,154][52833] Updated weights for policy 0, policy_version 1420 (0.0007) -[2023-10-15 14:51:32,223][52866] Updated weights for policy 1, policy_version 1430 (0.0007) -[2023-10-15 14:51:32,528][52833] Updated weights for policy 0, policy_version 1430 (0.0007) -[2023-10-15 14:51:32,595][52866] Updated weights for policy 1, policy_version 1440 (0.0008) -[2023-10-15 14:51:32,895][52833] Updated weights for policy 0, policy_version 1440 (0.0008) -[2023-10-15 14:51:33,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 13550.7). Total num frames: 2949120. Throughput: 0: 1752.4, 1: 1745.3. Samples: 737016. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 14:51:33,442][51532] Avg episode reward: [(0, '6.630'), (1, '6.540')] -[2023-10-15 14:51:36,431][52866] Updated weights for policy 1, policy_version 1450 (0.0008) -[2023-10-15 14:51:36,794][52866] Updated weights for policy 1, policy_version 1460 (0.0007) -[2023-10-15 14:51:36,924][52833] Updated weights for policy 0, policy_version 1450 (0.0008) -[2023-10-15 14:51:37,162][52866] Updated weights for policy 1, policy_version 1470 (0.0007) -[2023-10-15 14:51:37,297][52833] Updated weights for policy 0, policy_version 1460 (0.0009) -[2023-10-15 14:51:37,670][52833] Updated weights for policy 0, policy_version 1470 (0.0010) -[2023-10-15 14:51:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13540.8). Total num frames: 3014656. Throughput: 0: 1748.3, 1: 1731.6. Samples: 757366. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-15 14:51:38,442][51532] Avg episode reward: [(0, '6.960'), (1, '6.810')] -[2023-10-15 14:51:38,443][52410] Saving new best policy, reward=6.960! -[2023-10-15 14:51:40,876][52866] Updated weights for policy 1, policy_version 1480 (0.0009) -[2023-10-15 14:51:41,244][52866] Updated weights for policy 1, policy_version 1490 (0.0007) -[2023-10-15 14:51:41,557][52833] Updated weights for policy 0, policy_version 1480 (0.0008) -[2023-10-15 14:51:41,613][52866] Updated weights for policy 1, policy_version 1500 (0.0008) -[2023-10-15 14:51:41,929][52833] Updated weights for policy 0, policy_version 1490 (0.0007) -[2023-10-15 14:51:42,305][52833] Updated weights for policy 0, policy_version 1500 (0.0008) -[2023-10-15 14:51:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13531.3). Total num frames: 3080192. Throughput: 0: 1718.2, 1: 1725.5. Samples: 777776. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) -[2023-10-15 14:51:43,442][51532] Avg episode reward: [(0, '7.600'), (1, '6.890')] -[2023-10-15 14:51:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000001504_1540096.pth... -[2023-10-15 14:51:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000001504_1540096.pth... -[2023-10-15 14:51:43,486][52410] Saving new best policy, reward=7.600! -[2023-10-15 14:51:43,487][52518] Saving new best policy, reward=6.890! -[2023-10-15 14:51:45,484][52866] Updated weights for policy 1, policy_version 1510 (0.0007) -[2023-10-15 14:51:45,856][52866] Updated weights for policy 1, policy_version 1520 (0.0007) -[2023-10-15 14:51:46,204][52833] Updated weights for policy 0, policy_version 1510 (0.0010) -[2023-10-15 14:51:46,229][52866] Updated weights for policy 1, policy_version 1530 (0.0009) -[2023-10-15 14:51:46,572][52833] Updated weights for policy 0, policy_version 1520 (0.0008) -[2023-10-15 14:51:46,936][52833] Updated weights for policy 0, policy_version 1530 (0.0008) -[2023-10-15 14:51:48,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13522.2). Total num frames: 3145728. Throughput: 0: 1746.8, 1: 1738.9. Samples: 789264. Policy #0 lag: (min: 5.0, avg: 7.7, max: 37.0) -[2023-10-15 14:51:48,441][51532] Avg episode reward: [(0, '7.180'), (1, '6.500')] -[2023-10-15 14:51:50,216][52866] Updated weights for policy 1, policy_version 1540 (0.0008) -[2023-10-15 14:51:50,577][52866] Updated weights for policy 1, policy_version 1550 (0.0008) -[2023-10-15 14:51:50,861][52833] Updated weights for policy 0, policy_version 1540 (0.0009) -[2023-10-15 14:51:50,955][52866] Updated weights for policy 1, policy_version 1560 (0.0007) -[2023-10-15 14:51:51,222][52833] Updated weights for policy 0, policy_version 1550 (0.0008) -[2023-10-15 14:51:51,596][52833] Updated weights for policy 0, policy_version 1560 (0.0010) -[2023-10-15 14:51:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13513.4). Total num frames: 3211264. Throughput: 0: 1718.7, 1: 1719.7. Samples: 808566. Policy #0 lag: (min: 17.0, avg: 19.1, max: 49.0) -[2023-10-15 14:51:53,442][51532] Avg episode reward: [(0, '6.550'), (1, '6.970')] -[2023-10-15 14:51:53,442][52518] Saving new best policy, reward=6.970! -[2023-10-15 14:51:54,989][52866] Updated weights for policy 1, policy_version 1570 (0.0009) -[2023-10-15 14:51:55,230][52833] Updated weights for policy 0, policy_version 1570 (0.0008) -[2023-10-15 14:51:55,355][52866] Updated weights for policy 1, policy_version 1580 (0.0010) -[2023-10-15 14:51:55,592][52833] Updated weights for policy 0, policy_version 1580 (0.0007) -[2023-10-15 14:51:55,718][52866] Updated weights for policy 1, policy_version 1590 (0.0009) -[2023-10-15 14:51:55,959][52833] Updated weights for policy 0, policy_version 1590 (0.0007) -[2023-10-15 14:51:56,078][52866] Updated weights for policy 1, policy_version 1600 (0.0007) -[2023-10-15 14:51:56,331][52833] Updated weights for policy 0, policy_version 1600 (0.0009) -[2023-10-15 14:51:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13505.0). Total num frames: 3276800. Throughput: 0: 1725.1, 1: 1739.5. Samples: 830314. Policy #0 lag: (min: 30.0, avg: 31.3, max: 48.0) -[2023-10-15 14:51:58,442][51532] Avg episode reward: [(0, '7.460'), (1, '7.490')] -[2023-10-15 14:51:58,451][52518] Saving new best policy, reward=7.490! -[2023-10-15 14:52:00,046][52866] Updated weights for policy 1, policy_version 1610 (0.0007) -[2023-10-15 14:52:00,141][52833] Updated weights for policy 0, policy_version 1610 (0.0008) -[2023-10-15 14:52:00,402][52866] Updated weights for policy 1, policy_version 1620 (0.0008) -[2023-10-15 14:52:00,520][52833] Updated weights for policy 0, policy_version 1620 (0.0009) -[2023-10-15 14:52:00,773][52866] Updated weights for policy 1, policy_version 1630 (0.0008) -[2023-10-15 14:52:00,881][52833] Updated weights for policy 0, policy_version 1630 (0.0009) -[2023-10-15 14:52:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13497.0). Total num frames: 3342336. Throughput: 0: 1726.4, 1: 1717.7. Samples: 839740. Policy #0 lag: (min: 13.0, avg: 16.0, max: 45.0) -[2023-10-15 14:52:03,441][51532] Avg episode reward: [(0, '7.350'), (1, '7.400')] -[2023-10-15 14:52:04,783][52833] Updated weights for policy 0, policy_version 1640 (0.0008) -[2023-10-15 14:52:04,866][52866] Updated weights for policy 1, policy_version 1640 (0.0007) -[2023-10-15 14:52:05,152][52833] Updated weights for policy 0, policy_version 1650 (0.0009) -[2023-10-15 14:52:05,236][52866] Updated weights for policy 1, policy_version 1650 (0.0008) -[2023-10-15 14:52:05,518][52833] Updated weights for policy 0, policy_version 1660 (0.0008) -[2023-10-15 14:52:05,596][52866] Updated weights for policy 1, policy_version 1660 (0.0008) -[2023-10-15 14:52:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13489.3). Total num frames: 3407872. Throughput: 0: 1717.2, 1: 1724.6. Samples: 860876. Policy #0 lag: (min: 4.0, avg: 10.1, max: 36.0) -[2023-10-15 14:52:08,441][51532] Avg episode reward: [(0, '7.480'), (1, '6.260')] -[2023-10-15 14:52:09,512][52833] Updated weights for policy 0, policy_version 1670 (0.0008) -[2023-10-15 14:52:09,541][52866] Updated weights for policy 1, policy_version 1670 (0.0007) -[2023-10-15 14:52:09,876][52833] Updated weights for policy 0, policy_version 1680 (0.0007) -[2023-10-15 14:52:09,907][52866] Updated weights for policy 1, policy_version 1680 (0.0007) -[2023-10-15 14:52:10,248][52833] Updated weights for policy 0, policy_version 1690 (0.0007) -[2023-10-15 14:52:10,275][52866] Updated weights for policy 1, policy_version 1690 (0.0007) -[2023-10-15 14:52:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13481.9). Total num frames: 3473408. Throughput: 0: 1738.3, 1: 1753.9. Samples: 882422. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-15 14:52:13,442][51532] Avg episode reward: [(0, '7.100'), (1, '6.170')] -[2023-10-15 14:52:14,025][52866] Updated weights for policy 1, policy_version 1700 (0.0008) -[2023-10-15 14:52:14,223][52833] Updated weights for policy 0, policy_version 1700 (0.0008) -[2023-10-15 14:52:14,400][52866] Updated weights for policy 1, policy_version 1710 (0.0009) -[2023-10-15 14:52:14,588][52833] Updated weights for policy 0, policy_version 1710 (0.0009) -[2023-10-15 14:52:14,766][52866] Updated weights for policy 1, policy_version 1720 (0.0008) -[2023-10-15 14:52:14,968][52833] Updated weights for policy 0, policy_version 1720 (0.0010) -[2023-10-15 14:52:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13474.8). Total num frames: 3538944. Throughput: 0: 1720.2, 1: 1724.4. Samples: 892024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:52:18,441][51532] Avg episode reward: [(0, '7.260'), (1, '6.720')] -[2023-10-15 14:52:18,766][52866] Updated weights for policy 1, policy_version 1730 (0.0008) -[2023-10-15 14:52:18,821][52833] Updated weights for policy 0, policy_version 1730 (0.0007) -[2023-10-15 14:52:19,138][52866] Updated weights for policy 1, policy_version 1740 (0.0010) -[2023-10-15 14:52:19,179][52833] Updated weights for policy 0, policy_version 1740 (0.0008) -[2023-10-15 14:52:19,506][52866] Updated weights for policy 1, policy_version 1750 (0.0007) -[2023-10-15 14:52:19,557][52833] Updated weights for policy 0, policy_version 1750 (0.0009) -[2023-10-15 14:52:19,875][52866] Updated weights for policy 1, policy_version 1760 (0.0007) -[2023-10-15 14:52:19,939][52833] Updated weights for policy 0, policy_version 1760 (0.0009) -[2023-10-15 14:52:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13467.9). Total num frames: 3604480. Throughput: 0: 1725.7, 1: 1737.3. Samples: 913200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:52:23,442][51532] Avg episode reward: [(0, '7.320'), (1, '6.740')] -[2023-10-15 14:52:23,738][52866] Updated weights for policy 1, policy_version 1770 (0.0007) -[2023-10-15 14:52:23,909][52833] Updated weights for policy 0, policy_version 1770 (0.0007) -[2023-10-15 14:52:24,111][52866] Updated weights for policy 1, policy_version 1780 (0.0008) -[2023-10-15 14:52:24,271][52833] Updated weights for policy 0, policy_version 1780 (0.0009) -[2023-10-15 14:52:24,479][52866] Updated weights for policy 1, policy_version 1790 (0.0007) -[2023-10-15 14:52:24,638][52833] Updated weights for policy 0, policy_version 1790 (0.0008) -[2023-10-15 14:52:28,391][52866] Updated weights for policy 1, policy_version 1800 (0.0008) -[2023-10-15 14:52:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13461.3). Total num frames: 3670016. Throughput: 0: 1749.5, 1: 1741.0. Samples: 934848. Policy #0 lag: (min: 8.0, avg: 36.9, max: 40.0) -[2023-10-15 14:52:28,441][51532] Avg episode reward: [(0, '7.380'), (1, '7.280')] -[2023-10-15 14:52:28,606][52833] Updated weights for policy 0, policy_version 1800 (0.0007) -[2023-10-15 14:52:28,751][52866] Updated weights for policy 1, policy_version 1810 (0.0007) -[2023-10-15 14:52:28,973][52833] Updated weights for policy 0, policy_version 1810 (0.0009) -[2023-10-15 14:52:29,120][52866] Updated weights for policy 1, policy_version 1820 (0.0007) -[2023-10-15 14:52:29,342][52833] Updated weights for policy 0, policy_version 1820 (0.0008) -[2023-10-15 14:52:33,028][52866] Updated weights for policy 1, policy_version 1830 (0.0008) -[2023-10-15 14:52:33,262][52833] Updated weights for policy 0, policy_version 1830 (0.0008) -[2023-10-15 14:52:33,395][52866] Updated weights for policy 1, policy_version 1840 (0.0009) -[2023-10-15 14:52:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13454.9). Total num frames: 3735552. Throughput: 0: 1720.0, 1: 1726.0. Samples: 944332. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:52:33,441][51532] Avg episode reward: [(0, '7.840'), (1, '7.570')] -[2023-10-15 14:52:33,629][52833] Updated weights for policy 0, policy_version 1840 (0.0008) -[2023-10-15 14:52:33,762][52866] Updated weights for policy 1, policy_version 1850 (0.0009) -[2023-10-15 14:52:33,983][52518] Saving new best policy, reward=7.570! -[2023-10-15 14:52:33,992][52833] Updated weights for policy 0, policy_version 1850 (0.0007) -[2023-10-15 14:52:34,213][52410] Saving new best policy, reward=7.840! -[2023-10-15 14:52:37,728][52866] Updated weights for policy 1, policy_version 1860 (0.0009) -[2023-10-15 14:52:37,862][52833] Updated weights for policy 0, policy_version 1860 (0.0008) -[2023-10-15 14:52:38,096][52866] Updated weights for policy 1, policy_version 1870 (0.0008) -[2023-10-15 14:52:38,238][52833] Updated weights for policy 0, policy_version 1870 (0.0007) -[2023-10-15 14:52:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13448.7). Total num frames: 3801088. Throughput: 0: 1754.2, 1: 1738.8. Samples: 965752. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 14:52:38,441][51532] Avg episode reward: [(0, '7.180'), (1, '7.570')] -[2023-10-15 14:52:38,466][52866] Updated weights for policy 1, policy_version 1880 (0.0007) -[2023-10-15 14:52:38,606][52833] Updated weights for policy 0, policy_version 1880 (0.0007) -[2023-10-15 14:52:42,428][52866] Updated weights for policy 1, policy_version 1890 (0.0008) -[2023-10-15 14:52:42,479][52833] Updated weights for policy 0, policy_version 1890 (0.0008) -[2023-10-15 14:52:42,795][52866] Updated weights for policy 1, policy_version 1900 (0.0007) -[2023-10-15 14:52:42,841][52833] Updated weights for policy 0, policy_version 1900 (0.0007) -[2023-10-15 14:52:43,165][52866] Updated weights for policy 1, policy_version 1910 (0.0007) -[2023-10-15 14:52:43,208][52833] Updated weights for policy 0, policy_version 1910 (0.0007) -[2023-10-15 14:52:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13442.8). Total num frames: 3866624. Throughput: 0: 1736.9, 1: 1724.4. Samples: 986070. Policy #0 lag: (min: 26.0, avg: 28.1, max: 57.0) -[2023-10-15 14:52:43,441][51532] Avg episode reward: [(0, '7.610'), (1, '8.030')] -[2023-10-15 14:52:43,537][52518] Saving new best policy, reward=8.030! -[2023-10-15 14:52:43,537][52866] Updated weights for policy 1, policy_version 1920 (0.0008) -[2023-10-15 14:52:43,577][52833] Updated weights for policy 0, policy_version 1920 (0.0009) -[2023-10-15 14:52:47,436][52866] Updated weights for policy 1, policy_version 1930 (0.0009) -[2023-10-15 14:52:47,584][52833] Updated weights for policy 0, policy_version 1930 (0.0011) -[2023-10-15 14:52:47,797][52866] Updated weights for policy 1, policy_version 1940 (0.0008) -[2023-10-15 14:52:47,960][52833] Updated weights for policy 0, policy_version 1940 (0.0008) -[2023-10-15 14:52:48,159][52866] Updated weights for policy 1, policy_version 1950 (0.0007) -[2023-10-15 14:52:48,327][52833] Updated weights for policy 0, policy_version 1950 (0.0008) -[2023-10-15 14:52:48,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14199.5, 300 sec: 13661.0). Total num frames: 3997696. Throughput: 0: 1747.8, 1: 1740.4. Samples: 996712. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 14:52:48,442][51532] Avg episode reward: [(0, '8.120'), (1, '7.740')] -[2023-10-15 14:52:48,442][52410] Saving new best policy, reward=8.120! -[2023-10-15 14:52:52,254][52833] Updated weights for policy 0, policy_version 1960 (0.0010) -[2023-10-15 14:52:52,355][52866] Updated weights for policy 1, policy_version 1960 (0.0007) -[2023-10-15 14:52:52,624][52833] Updated weights for policy 0, policy_version 1970 (0.0008) -[2023-10-15 14:52:52,717][52866] Updated weights for policy 1, policy_version 1970 (0.0009) -[2023-10-15 14:52:52,992][52833] Updated weights for policy 0, policy_version 1980 (0.0008) -[2023-10-15 14:52:53,079][52866] Updated weights for policy 1, policy_version 1980 (0.0007) -[2023-10-15 14:52:53,441][51532] Fps is (10 sec: 19661.0, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 4063232. Throughput: 0: 1748.8, 1: 1735.3. Samples: 1017658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:52:53,441][51532] Avg episode reward: [(0, '8.050'), (1, '8.270')] -[2023-10-15 14:52:53,442][52518] Saving new best policy, reward=8.270! -[2023-10-15 14:52:56,788][52866] Updated weights for policy 1, policy_version 1990 (0.0008) -[2023-10-15 14:52:56,813][52833] Updated weights for policy 0, policy_version 1990 (0.0008) -[2023-10-15 14:52:57,162][52866] Updated weights for policy 1, policy_version 2000 (0.0009) -[2023-10-15 14:52:57,186][52833] Updated weights for policy 0, policy_version 2000 (0.0008) -[2023-10-15 14:52:57,528][52866] Updated weights for policy 1, policy_version 2010 (0.0008) -[2023-10-15 14:52:57,557][52833] Updated weights for policy 0, policy_version 2010 (0.0008) -[2023-10-15 14:52:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 4128768. Throughput: 0: 1717.1, 1: 1711.2. Samples: 1036694. Policy #0 lag: (min: 1.0, avg: 14.6, max: 33.0) -[2023-10-15 14:52:58,442][51532] Avg episode reward: [(0, '8.290'), (1, '8.450')] -[2023-10-15 14:52:58,452][52410] Saving new best policy, reward=8.290! -[2023-10-15 14:52:58,453][52518] Saving new best policy, reward=8.450! -[2023-10-15 14:53:01,346][52866] Updated weights for policy 1, policy_version 2020 (0.0010) -[2023-10-15 14:53:01,583][52833] Updated weights for policy 0, policy_version 2020 (0.0009) -[2023-10-15 14:53:01,709][52866] Updated weights for policy 1, policy_version 2030 (0.0007) -[2023-10-15 14:53:01,960][52833] Updated weights for policy 0, policy_version 2030 (0.0007) -[2023-10-15 14:53:02,077][52866] Updated weights for policy 1, policy_version 2040 (0.0007) -[2023-10-15 14:53:02,322][52833] Updated weights for policy 0, policy_version 2040 (0.0007) -[2023-10-15 14:53:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 4194304. Throughput: 0: 1745.7, 1: 1741.7. Samples: 1048958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:53:03,442][51532] Avg episode reward: [(0, '8.230'), (1, '7.960')] -[2023-10-15 14:53:06,018][52866] Updated weights for policy 1, policy_version 2050 (0.0007) -[2023-10-15 14:53:06,255][52833] Updated weights for policy 0, policy_version 2050 (0.0009) -[2023-10-15 14:53:06,385][52866] Updated weights for policy 1, policy_version 2060 (0.0008) -[2023-10-15 14:53:06,618][52833] Updated weights for policy 0, policy_version 2060 (0.0010) -[2023-10-15 14:53:06,753][52866] Updated weights for policy 1, policy_version 2070 (0.0009) -[2023-10-15 14:53:06,984][52833] Updated weights for policy 0, policy_version 2070 (0.0008) -[2023-10-15 14:53:07,126][52866] Updated weights for policy 1, policy_version 2080 (0.0008) -[2023-10-15 14:53:07,362][52833] Updated weights for policy 0, policy_version 2080 (0.0009) -[2023-10-15 14:53:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 4259840. Throughput: 0: 1732.0, 1: 1726.8. Samples: 1068848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:53:08,441][51532] Avg episode reward: [(0, '8.500'), (1, '7.620')] -[2023-10-15 14:53:08,442][52410] Saving new best policy, reward=8.500! -[2023-10-15 14:53:11,038][52866] Updated weights for policy 1, policy_version 2090 (0.0007) -[2023-10-15 14:53:11,270][52833] Updated weights for policy 0, policy_version 2090 (0.0010) -[2023-10-15 14:53:11,408][52866] Updated weights for policy 1, policy_version 2100 (0.0008) -[2023-10-15 14:53:11,648][52833] Updated weights for policy 0, policy_version 2100 (0.0009) -[2023-10-15 14:53:11,765][52866] Updated weights for policy 1, policy_version 2110 (0.0008) -[2023-10-15 14:53:12,020][52833] Updated weights for policy 0, policy_version 2110 (0.0008) -[2023-10-15 14:53:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 4325376. Throughput: 0: 1719.9, 1: 1723.2. Samples: 1089788. Policy #0 lag: (min: 26.0, avg: 26.3, max: 37.0) -[2023-10-15 14:53:13,442][51532] Avg episode reward: [(0, '8.840'), (1, '7.970')] -[2023-10-15 14:53:13,452][52410] Saving new best policy, reward=8.840! -[2023-10-15 14:53:15,460][52866] Updated weights for policy 1, policy_version 2120 (0.0008) -[2023-10-15 14:53:15,764][52833] Updated weights for policy 0, policy_version 2120 (0.0008) -[2023-10-15 14:53:15,829][52866] Updated weights for policy 1, policy_version 2130 (0.0007) -[2023-10-15 14:53:16,143][52833] Updated weights for policy 0, policy_version 2130 (0.0008) -[2023-10-15 14:53:16,186][52866] Updated weights for policy 1, policy_version 2140 (0.0009) -[2023-10-15 14:53:16,512][52833] Updated weights for policy 0, policy_version 2140 (0.0007) -[2023-10-15 14:53:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 4390912. Throughput: 0: 1742.3, 1: 1737.1. Samples: 1100906. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:53:18,442][51532] Avg episode reward: [(0, '8.700'), (1, '8.310')] -[2023-10-15 14:53:20,000][52866] Updated weights for policy 1, policy_version 2150 (0.0009) -[2023-10-15 14:53:20,262][52833] Updated weights for policy 0, policy_version 2150 (0.0009) -[2023-10-15 14:53:20,370][52866] Updated weights for policy 1, policy_version 2160 (0.0009) -[2023-10-15 14:53:20,633][52833] Updated weights for policy 0, policy_version 2160 (0.0008) -[2023-10-15 14:53:20,734][52866] Updated weights for policy 1, policy_version 2170 (0.0007) -[2023-10-15 14:53:21,008][52833] Updated weights for policy 0, policy_version 2170 (0.0007) -[2023-10-15 14:53:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 4456448. Throughput: 0: 1726.2, 1: 1730.8. Samples: 1121320. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) -[2023-10-15 14:53:23,442][51532] Avg episode reward: [(0, '9.050'), (1, '7.840')] -[2023-10-15 14:53:23,444][52410] Saving new best policy, reward=9.050! -[2023-10-15 14:53:24,548][52866] Updated weights for policy 1, policy_version 2180 (0.0008) -[2023-10-15 14:53:24,833][52833] Updated weights for policy 0, policy_version 2180 (0.0009) -[2023-10-15 14:53:24,914][52866] Updated weights for policy 1, policy_version 2190 (0.0010) -[2023-10-15 14:53:25,196][52833] Updated weights for policy 0, policy_version 2190 (0.0009) -[2023-10-15 14:53:25,281][52866] Updated weights for policy 1, policy_version 2200 (0.0007) -[2023-10-15 14:53:25,563][52833] Updated weights for policy 0, policy_version 2200 (0.0009) -[2023-10-15 14:53:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 4521984. Throughput: 0: 1740.3, 1: 1744.5. Samples: 1142886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:53:28,442][51532] Avg episode reward: [(0, '8.510'), (1, '8.000')] -[2023-10-15 14:53:29,288][52866] Updated weights for policy 1, policy_version 2210 (0.0008) -[2023-10-15 14:53:29,394][52833] Updated weights for policy 0, policy_version 2210 (0.0010) -[2023-10-15 14:53:29,658][52866] Updated weights for policy 1, policy_version 2220 (0.0008) -[2023-10-15 14:53:29,763][52833] Updated weights for policy 0, policy_version 2220 (0.0008) -[2023-10-15 14:53:30,022][52866] Updated weights for policy 1, policy_version 2230 (0.0009) -[2023-10-15 14:53:30,138][52833] Updated weights for policy 0, policy_version 2230 (0.0008) -[2023-10-15 14:53:30,390][52866] Updated weights for policy 1, policy_version 2240 (0.0009) -[2023-10-15 14:53:30,505][52833] Updated weights for policy 0, policy_version 2240 (0.0007) -[2023-10-15 14:53:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 4587520. Throughput: 0: 1730.6, 1: 1730.1. Samples: 1152444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:53:33,442][51532] Avg episode reward: [(0, '9.040'), (1, '8.240')] -[2023-10-15 14:53:34,288][52866] Updated weights for policy 1, policy_version 2250 (0.0009) -[2023-10-15 14:53:34,485][52833] Updated weights for policy 0, policy_version 2250 (0.0008) -[2023-10-15 14:53:34,657][52866] Updated weights for policy 1, policy_version 2260 (0.0008) -[2023-10-15 14:53:34,854][52833] Updated weights for policy 0, policy_version 2260 (0.0008) -[2023-10-15 14:53:35,021][52866] Updated weights for policy 1, policy_version 2270 (0.0009) -[2023-10-15 14:53:35,226][52833] Updated weights for policy 0, policy_version 2270 (0.0008) -[2023-10-15 14:53:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 4653056. Throughput: 0: 1734.5, 1: 1743.1. Samples: 1174152. Policy #0 lag: (min: 31.0, avg: 44.7, max: 63.0) -[2023-10-15 14:53:38,442][51532] Avg episode reward: [(0, '8.790'), (1, '8.650')] -[2023-10-15 14:53:38,791][52866] Updated weights for policy 1, policy_version 2280 (0.0008) -[2023-10-15 14:53:39,154][52866] Updated weights for policy 1, policy_version 2290 (0.0007) -[2023-10-15 14:53:39,187][52833] Updated weights for policy 0, policy_version 2280 (0.0008) -[2023-10-15 14:53:39,516][52866] Updated weights for policy 1, policy_version 2300 (0.0009) -[2023-10-15 14:53:39,567][52833] Updated weights for policy 0, policy_version 2290 (0.0008) -[2023-10-15 14:53:39,661][52518] Saving new best policy, reward=8.650! -[2023-10-15 14:53:39,935][52833] Updated weights for policy 0, policy_version 2300 (0.0007) -[2023-10-15 14:53:43,287][52866] Updated weights for policy 1, policy_version 2310 (0.0008) -[2023-10-15 14:53:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13773.7). Total num frames: 4718592. Throughput: 0: 1758.7, 1: 1775.4. Samples: 1195728. Policy #0 lag: (min: 9.0, avg: 19.0, max: 41.0) -[2023-10-15 14:53:43,442][51532] Avg episode reward: [(0, '8.870'), (1, '8.180')] -[2023-10-15 14:53:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000002304_2359296.pth... -[2023-10-15 14:53:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000000704_720896.pth -[2023-10-15 14:53:43,657][52866] Updated weights for policy 1, policy_version 2320 (0.0007) -[2023-10-15 14:53:43,991][52833] Updated weights for policy 0, policy_version 2310 (0.0009) -[2023-10-15 14:53:44,028][52866] Updated weights for policy 1, policy_version 2330 (0.0009) -[2023-10-15 14:53:44,251][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000002336_2392064.pth... -[2023-10-15 14:53:44,284][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000000704_720896.pth -[2023-10-15 14:53:44,359][52833] Updated weights for policy 0, policy_version 2320 (0.0008) -[2023-10-15 14:53:44,734][52833] Updated weights for policy 0, policy_version 2330 (0.0007) -[2023-10-15 14:53:47,879][52866] Updated weights for policy 1, policy_version 2340 (0.0009) -[2023-10-15 14:53:48,245][52866] Updated weights for policy 1, policy_version 2350 (0.0008) -[2023-10-15 14:53:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 4784128. Throughput: 0: 1726.0, 1: 1746.8. Samples: 1205232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:53:48,441][51532] Avg episode reward: [(0, '8.350'), (1, '8.520')] -[2023-10-15 14:53:48,613][52866] Updated weights for policy 1, policy_version 2360 (0.0010) -[2023-10-15 14:53:48,698][52833] Updated weights for policy 0, policy_version 2340 (0.0007) -[2023-10-15 14:53:49,057][52833] Updated weights for policy 0, policy_version 2350 (0.0010) -[2023-10-15 14:53:49,425][52833] Updated weights for policy 0, policy_version 2360 (0.0008) -[2023-10-15 14:53:52,690][52866] Updated weights for policy 1, policy_version 2370 (0.0010) -[2023-10-15 14:53:53,058][52866] Updated weights for policy 1, policy_version 2380 (0.0008) -[2023-10-15 14:53:53,378][52833] Updated weights for policy 0, policy_version 2370 (0.0008) -[2023-10-15 14:53:53,427][52866] Updated weights for policy 1, policy_version 2390 (0.0007) -[2023-10-15 14:53:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 4849664. Throughput: 0: 1742.5, 1: 1771.5. Samples: 1226976. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) -[2023-10-15 14:53:53,442][51532] Avg episode reward: [(0, '8.700'), (1, '7.990')] -[2023-10-15 14:53:53,750][52833] Updated weights for policy 0, policy_version 2380 (0.0009) -[2023-10-15 14:53:53,799][52866] Updated weights for policy 1, policy_version 2400 (0.0008) -[2023-10-15 14:53:54,114][52833] Updated weights for policy 0, policy_version 2390 (0.0008) -[2023-10-15 14:53:54,483][52833] Updated weights for policy 0, policy_version 2400 (0.0008) -[2023-10-15 14:53:57,652][52866] Updated weights for policy 1, policy_version 2410 (0.0009) -[2023-10-15 14:53:58,026][52866] Updated weights for policy 1, policy_version 2420 (0.0007) -[2023-10-15 14:53:58,395][52866] Updated weights for policy 1, policy_version 2430 (0.0008) -[2023-10-15 14:53:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13773.7). Total num frames: 4915200. Throughput: 0: 1758.7, 1: 1761.1. Samples: 1248178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:53:58,442][51532] Avg episode reward: [(0, '8.800'), (1, '8.630')] -[2023-10-15 14:53:58,505][52833] Updated weights for policy 0, policy_version 2410 (0.0007) -[2023-10-15 14:53:58,867][52833] Updated weights for policy 0, policy_version 2420 (0.0007) -[2023-10-15 14:53:59,248][52833] Updated weights for policy 0, policy_version 2430 (0.0007) -[2023-10-15 14:54:02,169][52866] Updated weights for policy 1, policy_version 2440 (0.0008) -[2023-10-15 14:54:02,540][52866] Updated weights for policy 1, policy_version 2450 (0.0010) -[2023-10-15 14:54:02,905][52866] Updated weights for policy 1, policy_version 2460 (0.0009) -[2023-10-15 14:54:03,171][52833] Updated weights for policy 0, policy_version 2440 (0.0009) -[2023-10-15 14:54:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 5013504. Throughput: 0: 1737.9, 1: 1768.2. Samples: 1258682. Policy #0 lag: (min: 4.0, avg: 10.1, max: 36.0) -[2023-10-15 14:54:03,442][51532] Avg episode reward: [(0, '9.020'), (1, '8.750')] -[2023-10-15 14:54:03,442][52518] Saving new best policy, reward=8.750! -[2023-10-15 14:54:03,537][52833] Updated weights for policy 0, policy_version 2450 (0.0008) -[2023-10-15 14:54:03,904][52833] Updated weights for policy 0, policy_version 2460 (0.0008) -[2023-10-15 14:54:06,782][52866] Updated weights for policy 1, policy_version 2470 (0.0008) -[2023-10-15 14:54:07,136][52866] Updated weights for policy 1, policy_version 2480 (0.0008) -[2023-10-15 14:54:07,507][52866] Updated weights for policy 1, policy_version 2490 (0.0008) -[2023-10-15 14:54:07,670][52833] Updated weights for policy 0, policy_version 2470 (0.0007) -[2023-10-15 14:54:08,054][52833] Updated weights for policy 0, policy_version 2480 (0.0008) -[2023-10-15 14:54:08,415][52833] Updated weights for policy 0, policy_version 2490 (0.0009) -[2023-10-15 14:54:08,441][51532] Fps is (10 sec: 16384.5, 60 sec: 13653.3, 300 sec: 13884.8). Total num frames: 5079040. Throughput: 0: 1750.5, 1: 1776.0. Samples: 1280010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:54:08,441][51532] Avg episode reward: [(0, '8.700'), (1, '8.840')] -[2023-10-15 14:54:08,442][52518] Saving new best policy, reward=8.840! -[2023-10-15 14:54:11,214][52866] Updated weights for policy 1, policy_version 2500 (0.0008) -[2023-10-15 14:54:11,578][52866] Updated weights for policy 1, policy_version 2510 (0.0008) -[2023-10-15 14:54:11,946][52866] Updated weights for policy 1, policy_version 2520 (0.0007) -[2023-10-15 14:54:12,360][52833] Updated weights for policy 0, policy_version 2500 (0.0008) -[2023-10-15 14:54:12,733][52833] Updated weights for policy 0, policy_version 2510 (0.0009) -[2023-10-15 14:54:13,098][52833] Updated weights for policy 0, policy_version 2520 (0.0010) -[2023-10-15 14:54:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 5177344. Throughput: 0: 1729.0, 1: 1768.2. Samples: 1300260. Policy #0 lag: (min: 18.0, avg: 27.4, max: 50.0) -[2023-10-15 14:54:13,442][51532] Avg episode reward: [(0, '8.360'), (1, '9.120')] -[2023-10-15 14:54:13,451][52518] Saving new best policy, reward=9.120! -[2023-10-15 14:54:15,631][52866] Updated weights for policy 1, policy_version 2530 (0.0009) -[2023-10-15 14:54:16,004][52866] Updated weights for policy 1, policy_version 2540 (0.0008) -[2023-10-15 14:54:16,370][52866] Updated weights for policy 1, policy_version 2550 (0.0008) -[2023-10-15 14:54:16,737][52866] Updated weights for policy 1, policy_version 2560 (0.0008) -[2023-10-15 14:54:17,042][52833] Updated weights for policy 0, policy_version 2530 (0.0011) -[2023-10-15 14:54:17,405][52833] Updated weights for policy 0, policy_version 2540 (0.0007) -[2023-10-15 14:54:17,787][52833] Updated weights for policy 0, policy_version 2550 (0.0008) -[2023-10-15 14:54:18,154][52833] Updated weights for policy 0, policy_version 2560 (0.0008) -[2023-10-15 14:54:18,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 5242880. Throughput: 0: 1743.7, 1: 1791.4. Samples: 1311524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:54:18,442][51532] Avg episode reward: [(0, '8.300'), (1, '9.010')] -[2023-10-15 14:54:20,497][52866] Updated weights for policy 1, policy_version 2570 (0.0008) -[2023-10-15 14:54:20,855][52866] Updated weights for policy 1, policy_version 2580 (0.0007) -[2023-10-15 14:54:21,221][52866] Updated weights for policy 1, policy_version 2590 (0.0009) -[2023-10-15 14:54:21,936][52833] Updated weights for policy 0, policy_version 2570 (0.0007) -[2023-10-15 14:54:22,306][52833] Updated weights for policy 0, policy_version 2580 (0.0007) -[2023-10-15 14:54:22,671][52833] Updated weights for policy 0, policy_version 2590 (0.0009) -[2023-10-15 14:54:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 5308416. Throughput: 0: 1741.6, 1: 1773.0. Samples: 1332310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:54:23,441][51532] Avg episode reward: [(0, '8.680'), (1, '8.960')] -[2023-10-15 14:54:25,023][52866] Updated weights for policy 1, policy_version 2600 (0.0009) -[2023-10-15 14:54:25,387][52866] Updated weights for policy 1, policy_version 2610 (0.0012) -[2023-10-15 14:54:25,758][52866] Updated weights for policy 1, policy_version 2620 (0.0009) -[2023-10-15 14:54:26,518][52833] Updated weights for policy 0, policy_version 2600 (0.0008) -[2023-10-15 14:54:26,893][52833] Updated weights for policy 0, policy_version 2610 (0.0008) -[2023-10-15 14:54:27,267][52833] Updated weights for policy 0, policy_version 2620 (0.0007) -[2023-10-15 14:54:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 5373952. Throughput: 0: 1727.8, 1: 1773.5. Samples: 1353286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 14:54:28,442][51532] Avg episode reward: [(0, '9.280'), (1, '10.040')] -[2023-10-15 14:54:28,455][52518] Saving new best policy, reward=10.040! -[2023-10-15 14:54:28,455][52410] Saving new best policy, reward=9.280! -[2023-10-15 14:54:29,495][52866] Updated weights for policy 1, policy_version 2630 (0.0009) -[2023-10-15 14:54:29,874][52866] Updated weights for policy 1, policy_version 2640 (0.0010) -[2023-10-15 14:54:30,235][52866] Updated weights for policy 1, policy_version 2650 (0.0008) -[2023-10-15 14:54:31,016][52833] Updated weights for policy 0, policy_version 2630 (0.0007) -[2023-10-15 14:54:31,379][52833] Updated weights for policy 0, policy_version 2640 (0.0010) -[2023-10-15 14:54:31,755][52833] Updated weights for policy 0, policy_version 2650 (0.0007) -[2023-10-15 14:54:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 5439488. Throughput: 0: 1760.2, 1: 1769.6. Samples: 1364072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 14:54:33,442][51532] Avg episode reward: [(0, '9.450'), (1, '9.260')] -[2023-10-15 14:54:33,443][52410] Saving new best policy, reward=9.450! -[2023-10-15 14:54:33,888][52866] Updated weights for policy 1, policy_version 2660 (0.0009) -[2023-10-15 14:54:34,250][52866] Updated weights for policy 1, policy_version 2670 (0.0009) -[2023-10-15 14:54:34,612][52866] Updated weights for policy 1, policy_version 2680 (0.0008) -[2023-10-15 14:54:35,497][52833] Updated weights for policy 0, policy_version 2660 (0.0007) -[2023-10-15 14:54:35,877][52833] Updated weights for policy 0, policy_version 2670 (0.0007) -[2023-10-15 14:54:36,241][52833] Updated weights for policy 0, policy_version 2680 (0.0007) -[2023-10-15 14:54:38,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 5505024. Throughput: 0: 1738.2, 1: 1781.0. Samples: 1385342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:54:38,441][51532] Avg episode reward: [(0, '9.560'), (1, '10.090')] -[2023-10-15 14:54:38,442][52410] Saving new best policy, reward=9.560! -[2023-10-15 14:54:38,544][52866] Updated weights for policy 1, policy_version 2690 (0.0007) -[2023-10-15 14:54:38,907][52866] Updated weights for policy 1, policy_version 2700 (0.0007) -[2023-10-15 14:54:39,277][52866] Updated weights for policy 1, policy_version 2710 (0.0008) -[2023-10-15 14:54:39,646][52866] Updated weights for policy 1, policy_version 2720 (0.0009) -[2023-10-15 14:54:39,647][52518] Saving new best policy, reward=10.090! -[2023-10-15 14:54:39,933][52833] Updated weights for policy 0, policy_version 2690 (0.0008) -[2023-10-15 14:54:40,303][52833] Updated weights for policy 0, policy_version 2700 (0.0009) -[2023-10-15 14:54:40,674][52833] Updated weights for policy 0, policy_version 2710 (0.0008) -[2023-10-15 14:54:41,039][52833] Updated weights for policy 0, policy_version 2720 (0.0007) -[2023-10-15 14:54:43,429][52866] Updated weights for policy 1, policy_version 2730 (0.0007) -[2023-10-15 14:54:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13884.7). Total num frames: 5570560. Throughput: 0: 1741.5, 1: 1798.8. Samples: 1407488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:54:43,441][51532] Avg episode reward: [(0, '9.000'), (1, '9.370')] -[2023-10-15 14:54:43,790][52866] Updated weights for policy 1, policy_version 2740 (0.0007) -[2023-10-15 14:54:44,156][52866] Updated weights for policy 1, policy_version 2750 (0.0008) -[2023-10-15 14:54:44,886][52833] Updated weights for policy 0, policy_version 2730 (0.0007) -[2023-10-15 14:54:45,262][52833] Updated weights for policy 0, policy_version 2740 (0.0008) -[2023-10-15 14:54:45,636][52833] Updated weights for policy 0, policy_version 2750 (0.0009) -[2023-10-15 14:54:48,044][52866] Updated weights for policy 1, policy_version 2760 (0.0007) -[2023-10-15 14:54:48,405][52866] Updated weights for policy 1, policy_version 2770 (0.0007) -[2023-10-15 14:54:48,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13884.7). Total num frames: 5636096. Throughput: 0: 1741.6, 1: 1778.1. Samples: 1417070. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) -[2023-10-15 14:54:48,442][51532] Avg episode reward: [(0, '9.220'), (1, '9.200')] -[2023-10-15 14:54:48,778][52866] Updated weights for policy 1, policy_version 2780 (0.0010) -[2023-10-15 14:54:49,579][52833] Updated weights for policy 0, policy_version 2760 (0.0009) -[2023-10-15 14:54:49,950][52833] Updated weights for policy 0, policy_version 2770 (0.0009) -[2023-10-15 14:54:50,330][52833] Updated weights for policy 0, policy_version 2780 (0.0008) -[2023-10-15 14:54:52,509][52866] Updated weights for policy 1, policy_version 2790 (0.0008) -[2023-10-15 14:54:52,871][52866] Updated weights for policy 1, policy_version 2800 (0.0007) -[2023-10-15 14:54:53,249][52866] Updated weights for policy 1, policy_version 2810 (0.0008) -[2023-10-15 14:54:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13773.7). Total num frames: 5701632. Throughput: 0: 1751.5, 1: 1788.4. Samples: 1439304. Policy #0 lag: (min: 25.0, avg: 34.7, max: 57.0) -[2023-10-15 14:54:53,441][51532] Avg episode reward: [(0, '9.030'), (1, '9.080')] -[2023-10-15 14:54:54,128][52833] Updated weights for policy 0, policy_version 2790 (0.0010) -[2023-10-15 14:54:54,508][52833] Updated weights for policy 0, policy_version 2800 (0.0007) -[2023-10-15 14:54:54,874][52833] Updated weights for policy 0, policy_version 2810 (0.0009) -[2023-10-15 14:54:57,024][52866] Updated weights for policy 1, policy_version 2820 (0.0008) -[2023-10-15 14:54:57,391][52866] Updated weights for policy 1, policy_version 2830 (0.0007) -[2023-10-15 14:54:57,757][52866] Updated weights for policy 1, policy_version 2840 (0.0009) -[2023-10-15 14:54:58,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 13884.7). Total num frames: 5799936. Throughput: 0: 1774.7, 1: 1777.3. Samples: 1460102. Policy #0 lag: (min: 19.0, avg: 19.1, max: 26.0) -[2023-10-15 14:54:58,442][51532] Avg episode reward: [(0, '8.750'), (1, '9.580')] -[2023-10-15 14:54:58,776][52833] Updated weights for policy 0, policy_version 2820 (0.0009) -[2023-10-15 14:54:59,142][52833] Updated weights for policy 0, policy_version 2830 (0.0010) -[2023-10-15 14:54:59,516][52833] Updated weights for policy 0, policy_version 2840 (0.0009) -[2023-10-15 14:55:01,627][52866] Updated weights for policy 1, policy_version 2850 (0.0008) -[2023-10-15 14:55:01,995][52866] Updated weights for policy 1, policy_version 2860 (0.0007) -[2023-10-15 14:55:02,363][52866] Updated weights for policy 1, policy_version 2870 (0.0010) -[2023-10-15 14:55:02,731][52866] Updated weights for policy 1, policy_version 2880 (0.0010) -[2023-10-15 14:55:03,262][52833] Updated weights for policy 0, policy_version 2850 (0.0007) -[2023-10-15 14:55:03,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 5865472. Throughput: 0: 1754.7, 1: 1787.0. Samples: 1470902. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 14:55:03,442][51532] Avg episode reward: [(0, '9.200'), (1, '9.440')] -[2023-10-15 14:55:03,627][52833] Updated weights for policy 0, policy_version 2860 (0.0009) -[2023-10-15 14:55:04,000][52833] Updated weights for policy 0, policy_version 2870 (0.0008) -[2023-10-15 14:55:04,377][52833] Updated weights for policy 0, policy_version 2880 (0.0009) -[2023-10-15 14:55:06,568][52866] Updated weights for policy 1, policy_version 2890 (0.0009) -[2023-10-15 14:55:06,939][52866] Updated weights for policy 1, policy_version 2900 (0.0009) -[2023-10-15 14:55:07,315][52866] Updated weights for policy 1, policy_version 2910 (0.0007) -[2023-10-15 14:55:08,188][52833] Updated weights for policy 0, policy_version 2890 (0.0008) -[2023-10-15 14:55:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13884.8). Total num frames: 5931008. Throughput: 0: 1762.8, 1: 1783.9. Samples: 1491908. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 14:55:08,441][51532] Avg episode reward: [(0, '9.540'), (1, '10.140')] -[2023-10-15 14:55:08,442][52518] Saving new best policy, reward=10.140! -[2023-10-15 14:55:08,566][52833] Updated weights for policy 0, policy_version 2900 (0.0007) -[2023-10-15 14:55:08,940][52833] Updated weights for policy 0, policy_version 2910 (0.0007) -[2023-10-15 14:55:11,192][52866] Updated weights for policy 1, policy_version 2920 (0.0009) -[2023-10-15 14:55:11,572][52866] Updated weights for policy 1, policy_version 2930 (0.0010) -[2023-10-15 14:55:11,937][52866] Updated weights for policy 1, policy_version 2940 (0.0008) -[2023-10-15 14:55:12,764][52833] Updated weights for policy 0, policy_version 2920 (0.0009) -[2023-10-15 14:55:13,135][52833] Updated weights for policy 0, policy_version 2930 (0.0011) -[2023-10-15 14:55:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13884.7). Total num frames: 5996544. Throughput: 0: 1776.1, 1: 1770.3. Samples: 1512874. Policy #0 lag: (min: 15.0, avg: 20.3, max: 47.0) -[2023-10-15 14:55:13,442][51532] Avg episode reward: [(0, '9.380'), (1, '10.110')] -[2023-10-15 14:55:13,512][52833] Updated weights for policy 0, policy_version 2940 (0.0010) -[2023-10-15 14:55:15,635][52866] Updated weights for policy 1, policy_version 2950 (0.0008) -[2023-10-15 14:55:16,002][52866] Updated weights for policy 1, policy_version 2960 (0.0009) -[2023-10-15 14:55:16,379][52866] Updated weights for policy 1, policy_version 2970 (0.0008) -[2023-10-15 14:55:17,375][52833] Updated weights for policy 0, policy_version 2950 (0.0007) -[2023-10-15 14:55:17,752][52833] Updated weights for policy 0, policy_version 2960 (0.0007) -[2023-10-15 14:55:18,110][52833] Updated weights for policy 0, policy_version 2970 (0.0008) -[2023-10-15 14:55:18,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 6094848. Throughput: 0: 1757.9, 1: 1789.3. Samples: 1523694. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 14:55:18,441][51532] Avg episode reward: [(0, '10.060'), (1, '9.850')] -[2023-10-15 14:55:18,442][52410] Saving new best policy, reward=10.060! -[2023-10-15 14:55:20,091][52866] Updated weights for policy 1, policy_version 2980 (0.0010) -[2023-10-15 14:55:20,456][52866] Updated weights for policy 1, policy_version 2990 (0.0010) -[2023-10-15 14:55:20,822][52866] Updated weights for policy 1, policy_version 3000 (0.0009) -[2023-10-15 14:55:21,999][52833] Updated weights for policy 0, policy_version 2980 (0.0008) -[2023-10-15 14:55:22,368][52833] Updated weights for policy 0, policy_version 2990 (0.0009) -[2023-10-15 14:55:22,740][52833] Updated weights for policy 0, policy_version 3000 (0.0008) -[2023-10-15 14:55:23,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 6160384. Throughput: 0: 1778.3, 1: 1764.0. Samples: 1544746. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 14:55:23,441][51532] Avg episode reward: [(0, '10.060'), (1, '9.680')] -[2023-10-15 14:55:24,614][52866] Updated weights for policy 1, policy_version 3010 (0.0007) -[2023-10-15 14:55:24,996][52866] Updated weights for policy 1, policy_version 3020 (0.0009) -[2023-10-15 14:55:25,365][52866] Updated weights for policy 1, policy_version 3030 (0.0011) -[2023-10-15 14:55:25,741][52866] Updated weights for policy 1, policy_version 3040 (0.0010) -[2023-10-15 14:55:26,621][52833] Updated weights for policy 0, policy_version 3010 (0.0008) -[2023-10-15 14:55:26,985][52833] Updated weights for policy 0, policy_version 3020 (0.0008) -[2023-10-15 14:55:27,360][52833] Updated weights for policy 0, policy_version 3030 (0.0008) -[2023-10-15 14:55:27,730][52833] Updated weights for policy 0, policy_version 3040 (0.0009) -[2023-10-15 14:55:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 6225920. Throughput: 0: 1745.3, 1: 1761.8. Samples: 1565306. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) -[2023-10-15 14:55:28,441][51532] Avg episode reward: [(0, '10.070'), (1, '9.850')] -[2023-10-15 14:55:28,451][52410] Saving new best policy, reward=10.070! -[2023-10-15 14:55:29,619][52866] Updated weights for policy 1, policy_version 3050 (0.0009) -[2023-10-15 14:55:29,983][52866] Updated weights for policy 1, policy_version 3060 (0.0009) -[2023-10-15 14:55:30,349][52866] Updated weights for policy 1, policy_version 3070 (0.0010) -[2023-10-15 14:55:31,706][52833] Updated weights for policy 0, policy_version 3050 (0.0007) -[2023-10-15 14:55:32,073][52833] Updated weights for policy 0, policy_version 3060 (0.0008) -[2023-10-15 14:55:32,451][52833] Updated weights for policy 0, policy_version 3070 (0.0007) -[2023-10-15 14:55:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 6291456. Throughput: 0: 1773.4, 1: 1760.8. Samples: 1576112. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) -[2023-10-15 14:55:33,442][51532] Avg episode reward: [(0, '9.300'), (1, '9.460')] -[2023-10-15 14:55:34,130][52866] Updated weights for policy 1, policy_version 3080 (0.0010) -[2023-10-15 14:55:34,495][52866] Updated weights for policy 1, policy_version 3090 (0.0008) -[2023-10-15 14:55:34,861][52866] Updated weights for policy 1, policy_version 3100 (0.0009) -[2023-10-15 14:55:36,301][52833] Updated weights for policy 0, policy_version 3080 (0.0009) -[2023-10-15 14:55:36,664][52833] Updated weights for policy 0, policy_version 3090 (0.0008) -[2023-10-15 14:55:37,037][52833] Updated weights for policy 0, policy_version 3100 (0.0008) -[2023-10-15 14:55:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 6356992. Throughput: 0: 1749.8, 1: 1756.5. Samples: 1597088. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 14:55:38,442][51532] Avg episode reward: [(0, '9.440'), (1, '9.660')] -[2023-10-15 14:55:38,813][52866] Updated weights for policy 1, policy_version 3110 (0.0007) -[2023-10-15 14:55:39,177][52866] Updated weights for policy 1, policy_version 3120 (0.0007) -[2023-10-15 14:55:39,538][52866] Updated weights for policy 1, policy_version 3130 (0.0010) -[2023-10-15 14:55:40,939][52833] Updated weights for policy 0, policy_version 3110 (0.0011) -[2023-10-15 14:55:41,309][52833] Updated weights for policy 0, policy_version 3120 (0.0009) -[2023-10-15 14:55:41,683][52833] Updated weights for policy 0, policy_version 3130 (0.0010) -[2023-10-15 14:55:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 6422528. Throughput: 0: 1737.5, 1: 1778.7. Samples: 1618330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:55:43,441][51532] Avg episode reward: [(0, '9.560'), (1, '10.850')] -[2023-10-15 14:55:43,449][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000003136_3211264.pth... -[2023-10-15 14:55:43,486][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000001504_1540096.pth -[2023-10-15 14:55:43,500][52866] Updated weights for policy 1, policy_version 3140 (0.0007) -[2023-10-15 14:55:43,860][52866] Updated weights for policy 1, policy_version 3150 (0.0008) -[2023-10-15 14:55:44,234][52866] Updated weights for policy 1, policy_version 3160 (0.0010) -[2023-10-15 14:55:44,517][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000003168_3244032.pth... -[2023-10-15 14:55:44,556][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000001504_1540096.pth -[2023-10-15 14:55:44,560][52518] Saving new best policy, reward=10.850! -[2023-10-15 14:55:45,508][52833] Updated weights for policy 0, policy_version 3140 (0.0009) -[2023-10-15 14:55:45,886][52833] Updated weights for policy 0, policy_version 3150 (0.0008) -[2023-10-15 14:55:46,258][52833] Updated weights for policy 0, policy_version 3160 (0.0011) -[2023-10-15 14:55:48,210][52866] Updated weights for policy 1, policy_version 3170 (0.0008) -[2023-10-15 14:55:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 6488064. Throughput: 0: 1761.7, 1: 1746.6. Samples: 1628778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:55:48,441][51532] Avg episode reward: [(0, '9.780'), (1, '9.700')] -[2023-10-15 14:55:48,587][52866] Updated weights for policy 1, policy_version 3180 (0.0008) -[2023-10-15 14:55:48,944][52866] Updated weights for policy 1, policy_version 3190 (0.0007) -[2023-10-15 14:55:49,322][52866] Updated weights for policy 1, policy_version 3200 (0.0007) -[2023-10-15 14:55:50,074][52833] Updated weights for policy 0, policy_version 3170 (0.0010) -[2023-10-15 14:55:50,443][52833] Updated weights for policy 0, policy_version 3180 (0.0007) -[2023-10-15 14:55:50,803][52833] Updated weights for policy 0, policy_version 3190 (0.0007) -[2023-10-15 14:55:51,181][52833] Updated weights for policy 0, policy_version 3200 (0.0009) -[2023-10-15 14:55:53,064][52866] Updated weights for policy 1, policy_version 3210 (0.0008) -[2023-10-15 14:55:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 6553600. Throughput: 0: 1739.8, 1: 1765.2. Samples: 1649634. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 14:55:53,441][51532] Avg episode reward: [(0, '9.210'), (1, '9.870')] -[2023-10-15 14:55:53,442][52866] Updated weights for policy 1, policy_version 3220 (0.0009) -[2023-10-15 14:55:53,801][52866] Updated weights for policy 1, policy_version 3230 (0.0011) -[2023-10-15 14:55:55,096][52833] Updated weights for policy 0, policy_version 3210 (0.0009) -[2023-10-15 14:55:55,467][52833] Updated weights for policy 0, policy_version 3220 (0.0008) -[2023-10-15 14:55:55,832][52833] Updated weights for policy 0, policy_version 3230 (0.0009) -[2023-10-15 14:55:57,499][52866] Updated weights for policy 1, policy_version 3240 (0.0008) -[2023-10-15 14:55:57,860][52866] Updated weights for policy 1, policy_version 3250 (0.0007) -[2023-10-15 14:55:58,230][52866] Updated weights for policy 1, policy_version 3260 (0.0007) -[2023-10-15 14:55:58,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 6651904. Throughput: 0: 1750.1, 1: 1762.2. Samples: 1670930. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 14:55:58,442][51532] Avg episode reward: [(0, '9.530'), (1, '9.510')] -[2023-10-15 14:55:59,568][52833] Updated weights for policy 0, policy_version 3240 (0.0009) -[2023-10-15 14:55:59,951][52833] Updated weights for policy 0, policy_version 3250 (0.0009) -[2023-10-15 14:56:00,322][52833] Updated weights for policy 0, policy_version 3260 (0.0009) -[2023-10-15 14:56:02,069][52866] Updated weights for policy 1, policy_version 3270 (0.0007) -[2023-10-15 14:56:02,443][52866] Updated weights for policy 1, policy_version 3280 (0.0008) -[2023-10-15 14:56:02,806][52866] Updated weights for policy 1, policy_version 3290 (0.0008) -[2023-10-15 14:56:03,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 6717440. Throughput: 0: 1736.5, 1: 1765.1. Samples: 1681266. Policy #0 lag: (min: 19.0, avg: 23.2, max: 51.0) -[2023-10-15 14:56:03,441][51532] Avg episode reward: [(0, '9.290'), (1, '10.660')] -[2023-10-15 14:56:04,103][52833] Updated weights for policy 0, policy_version 3270 (0.0009) -[2023-10-15 14:56:04,473][52833] Updated weights for policy 0, policy_version 3280 (0.0008) -[2023-10-15 14:56:04,838][52833] Updated weights for policy 0, policy_version 3290 (0.0009) -[2023-10-15 14:56:06,678][52866] Updated weights for policy 1, policy_version 3300 (0.0008) -[2023-10-15 14:56:07,041][52866] Updated weights for policy 1, policy_version 3310 (0.0008) -[2023-10-15 14:56:07,409][52866] Updated weights for policy 1, policy_version 3320 (0.0007) -[2023-10-15 14:56:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 6782976. Throughput: 0: 1746.5, 1: 1769.3. Samples: 1702958. Policy #0 lag: (min: 19.0, avg: 23.2, max: 51.0) -[2023-10-15 14:56:08,442][51532] Avg episode reward: [(0, '10.180'), (1, '10.830')] -[2023-10-15 14:56:08,514][52833] Updated weights for policy 0, policy_version 3300 (0.0009) -[2023-10-15 14:56:08,885][52833] Updated weights for policy 0, policy_version 3310 (0.0011) -[2023-10-15 14:56:09,245][52833] Updated weights for policy 0, policy_version 3320 (0.0007) -[2023-10-15 14:56:09,544][52410] Saving new best policy, reward=10.180! -[2023-10-15 14:56:11,188][52866] Updated weights for policy 1, policy_version 3330 (0.0011) -[2023-10-15 14:56:11,561][52866] Updated weights for policy 1, policy_version 3340 (0.0008) -[2023-10-15 14:56:11,935][52866] Updated weights for policy 1, policy_version 3350 (0.0009) -[2023-10-15 14:56:12,309][52866] Updated weights for policy 1, policy_version 3360 (0.0008) -[2023-10-15 14:56:13,100][52833] Updated weights for policy 0, policy_version 3330 (0.0008) -[2023-10-15 14:56:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 6848512. Throughput: 0: 1784.1, 1: 1750.6. Samples: 1724368. Policy #0 lag: (min: 31.0, avg: 31.8, max: 49.0) -[2023-10-15 14:56:13,442][51532] Avg episode reward: [(0, '10.420'), (1, '10.240')] -[2023-10-15 14:56:13,476][52833] Updated weights for policy 0, policy_version 3340 (0.0008) -[2023-10-15 14:56:13,840][52833] Updated weights for policy 0, policy_version 3350 (0.0009) -[2023-10-15 14:56:14,208][52410] Saving new best policy, reward=10.420! -[2023-10-15 14:56:14,213][52833] Updated weights for policy 0, policy_version 3360 (0.0008) -[2023-10-15 14:56:16,106][52866] Updated weights for policy 1, policy_version 3370 (0.0008) -[2023-10-15 14:56:16,477][52866] Updated weights for policy 1, policy_version 3380 (0.0008) -[2023-10-15 14:56:16,839][52866] Updated weights for policy 1, policy_version 3390 (0.0008) -[2023-10-15 14:56:18,200][52833] Updated weights for policy 0, policy_version 3370 (0.0009) -[2023-10-15 14:56:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13995.8). Total num frames: 6914048. Throughput: 0: 1754.8, 1: 1780.7. Samples: 1735210. Policy #0 lag: (min: 31.0, avg: 31.8, max: 49.0) -[2023-10-15 14:56:18,442][51532] Avg episode reward: [(0, '9.990'), (1, '10.250')] -[2023-10-15 14:56:18,586][52833] Updated weights for policy 0, policy_version 3380 (0.0009) -[2023-10-15 14:56:18,960][52833] Updated weights for policy 0, policy_version 3390 (0.0010) -[2023-10-15 14:56:20,658][52866] Updated weights for policy 1, policy_version 3400 (0.0008) -[2023-10-15 14:56:21,033][52866] Updated weights for policy 1, policy_version 3410 (0.0007) -[2023-10-15 14:56:21,400][52866] Updated weights for policy 1, policy_version 3420 (0.0010) -[2023-10-15 14:56:22,786][52833] Updated weights for policy 0, policy_version 3400 (0.0010) -[2023-10-15 14:56:23,156][52833] Updated weights for policy 0, policy_version 3410 (0.0009) -[2023-10-15 14:56:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13884.8). Total num frames: 6979584. Throughput: 0: 1771.3, 1: 1757.2. Samples: 1755872. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) -[2023-10-15 14:56:23,442][51532] Avg episode reward: [(0, '10.770'), (1, '10.070')] -[2023-10-15 14:56:23,526][52833] Updated weights for policy 0, policy_version 3420 (0.0007) -[2023-10-15 14:56:23,670][52410] Saving new best policy, reward=10.770! -[2023-10-15 14:56:25,109][52866] Updated weights for policy 1, policy_version 3430 (0.0009) -[2023-10-15 14:56:25,484][52866] Updated weights for policy 1, policy_version 3440 (0.0009) -[2023-10-15 14:56:25,851][52866] Updated weights for policy 1, policy_version 3450 (0.0008) -[2023-10-15 14:56:27,392][52833] Updated weights for policy 0, policy_version 3430 (0.0009) -[2023-10-15 14:56:27,758][52833] Updated weights for policy 0, policy_version 3440 (0.0011) -[2023-10-15 14:56:28,131][52833] Updated weights for policy 0, policy_version 3450 (0.0007) -[2023-10-15 14:56:28,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 7077888. Throughput: 0: 1765.2, 1: 1764.6. Samples: 1777170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:56:28,442][51532] Avg episode reward: [(0, '10.070'), (1, '10.470')] -[2023-10-15 14:56:29,620][52866] Updated weights for policy 1, policy_version 3460 (0.0007) -[2023-10-15 14:56:29,985][52866] Updated weights for policy 1, policy_version 3470 (0.0009) -[2023-10-15 14:56:30,351][52866] Updated weights for policy 1, policy_version 3480 (0.0010) -[2023-10-15 14:56:32,095][52833] Updated weights for policy 0, policy_version 3460 (0.0010) -[2023-10-15 14:56:32,466][52833] Updated weights for policy 0, policy_version 3470 (0.0008) -[2023-10-15 14:56:32,839][52833] Updated weights for policy 0, policy_version 3480 (0.0008) -[2023-10-15 14:56:33,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 7143424. Throughput: 0: 1761.9, 1: 1768.7. Samples: 1787656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:56:33,442][51532] Avg episode reward: [(0, '9.830'), (1, '10.830')] -[2023-10-15 14:56:34,283][52866] Updated weights for policy 1, policy_version 3490 (0.0009) -[2023-10-15 14:56:34,657][52866] Updated weights for policy 1, policy_version 3500 (0.0007) -[2023-10-15 14:56:35,025][52866] Updated weights for policy 1, policy_version 3510 (0.0008) -[2023-10-15 14:56:35,392][52866] Updated weights for policy 1, policy_version 3520 (0.0008) -[2023-10-15 14:56:36,696][52833] Updated weights for policy 0, policy_version 3490 (0.0008) -[2023-10-15 14:56:37,059][52833] Updated weights for policy 0, policy_version 3500 (0.0008) -[2023-10-15 14:56:37,445][52833] Updated weights for policy 0, policy_version 3510 (0.0008) -[2023-10-15 14:56:37,803][52833] Updated weights for policy 0, policy_version 3520 (0.0008) -[2023-10-15 14:56:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 7208960. Throughput: 0: 1776.0, 1: 1767.6. Samples: 1809092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:56:38,441][51532] Avg episode reward: [(0, '9.220'), (1, '10.890')] -[2023-10-15 14:56:38,442][52518] Saving new best policy, reward=10.890! -[2023-10-15 14:56:39,370][52866] Updated weights for policy 1, policy_version 3530 (0.0007) -[2023-10-15 14:56:39,741][52866] Updated weights for policy 1, policy_version 3540 (0.0008) -[2023-10-15 14:56:40,107][52866] Updated weights for policy 1, policy_version 3550 (0.0007) -[2023-10-15 14:56:41,393][52833] Updated weights for policy 0, policy_version 3530 (0.0007) -[2023-10-15 14:56:41,765][52833] Updated weights for policy 0, policy_version 3540 (0.0007) -[2023-10-15 14:56:42,141][52833] Updated weights for policy 0, policy_version 3550 (0.0007) -[2023-10-15 14:56:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13995.8). Total num frames: 7274496. Throughput: 0: 1755.9, 1: 1788.0. Samples: 1830410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:56:43,442][51532] Avg episode reward: [(0, '10.080'), (1, '10.720')] -[2023-10-15 14:56:43,954][52866] Updated weights for policy 1, policy_version 3560 (0.0009) -[2023-10-15 14:56:44,312][52866] Updated weights for policy 1, policy_version 3570 (0.0007) -[2023-10-15 14:56:44,684][52866] Updated weights for policy 1, policy_version 3580 (0.0008) -[2023-10-15 14:56:46,035][52833] Updated weights for policy 0, policy_version 3560 (0.0007) -[2023-10-15 14:56:46,399][52833] Updated weights for policy 0, policy_version 3570 (0.0011) -[2023-10-15 14:56:46,770][52833] Updated weights for policy 0, policy_version 3580 (0.0009) -[2023-10-15 14:56:48,379][52866] Updated weights for policy 1, policy_version 3590 (0.0008) -[2023-10-15 14:56:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 7340032. Throughput: 0: 1786.5, 1: 1768.3. Samples: 1841232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:56:48,441][51532] Avg episode reward: [(0, '10.290'), (1, '10.620')] -[2023-10-15 14:56:48,756][52866] Updated weights for policy 1, policy_version 3600 (0.0009) -[2023-10-15 14:56:49,114][52866] Updated weights for policy 1, policy_version 3610 (0.0008) -[2023-10-15 14:56:50,543][52833] Updated weights for policy 0, policy_version 3590 (0.0007) -[2023-10-15 14:56:50,912][52833] Updated weights for policy 0, policy_version 3600 (0.0008) -[2023-10-15 14:56:51,282][52833] Updated weights for policy 0, policy_version 3610 (0.0010) -[2023-10-15 14:56:52,895][52866] Updated weights for policy 1, policy_version 3620 (0.0008) -[2023-10-15 14:56:53,269][52866] Updated weights for policy 1, policy_version 3630 (0.0007) -[2023-10-15 14:56:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 7405568. Throughput: 0: 1756.8, 1: 1784.0. Samples: 1862294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:56:53,441][51532] Avg episode reward: [(0, '10.350'), (1, '10.430')] -[2023-10-15 14:56:53,633][52866] Updated weights for policy 1, policy_version 3640 (0.0008) -[2023-10-15 14:56:54,901][52833] Updated weights for policy 0, policy_version 3620 (0.0008) -[2023-10-15 14:56:55,269][52833] Updated weights for policy 0, policy_version 3630 (0.0008) -[2023-10-15 14:56:55,639][52833] Updated weights for policy 0, policy_version 3640 (0.0011) -[2023-10-15 14:56:57,387][52866] Updated weights for policy 1, policy_version 3650 (0.0010) -[2023-10-15 14:56:57,761][52866] Updated weights for policy 1, policy_version 3660 (0.0007) -[2023-10-15 14:56:58,132][52866] Updated weights for policy 1, policy_version 3670 (0.0007) -[2023-10-15 14:56:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13995.8). Total num frames: 7471104. Throughput: 0: 1754.5, 1: 1783.3. Samples: 1883570. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:56:58,441][51532] Avg episode reward: [(0, '10.380'), (1, '10.670')] -[2023-10-15 14:56:58,502][52866] Updated weights for policy 1, policy_version 3680 (0.0008) -[2023-10-15 14:56:59,534][52833] Updated weights for policy 0, policy_version 3650 (0.0009) -[2023-10-15 14:56:59,902][52833] Updated weights for policy 0, policy_version 3660 (0.0007) -[2023-10-15 14:57:00,269][52833] Updated weights for policy 0, policy_version 3670 (0.0009) -[2023-10-15 14:57:00,634][52833] Updated weights for policy 0, policy_version 3680 (0.0010) -[2023-10-15 14:57:02,256][52866] Updated weights for policy 1, policy_version 3690 (0.0007) -[2023-10-15 14:57:02,618][52866] Updated weights for policy 1, policy_version 3700 (0.0008) -[2023-10-15 14:57:02,985][52866] Updated weights for policy 1, policy_version 3710 (0.0008) -[2023-10-15 14:57:03,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 7569408. Throughput: 0: 1754.2, 1: 1776.8. Samples: 1894104. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:57:03,442][51532] Avg episode reward: [(0, '10.630'), (1, '9.980')] -[2023-10-15 14:57:04,563][52833] Updated weights for policy 0, policy_version 3690 (0.0010) -[2023-10-15 14:57:04,934][52833] Updated weights for policy 0, policy_version 3700 (0.0008) -[2023-10-15 14:57:05,309][52833] Updated weights for policy 0, policy_version 3710 (0.0007) -[2023-10-15 14:57:06,564][52866] Updated weights for policy 1, policy_version 3720 (0.0007) -[2023-10-15 14:57:06,933][52866] Updated weights for policy 1, policy_version 3730 (0.0010) -[2023-10-15 14:57:07,302][52866] Updated weights for policy 1, policy_version 3740 (0.0011) -[2023-10-15 14:57:08,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 7634944. Throughput: 0: 1759.5, 1: 1789.6. Samples: 1915580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:57:08,442][51532] Avg episode reward: [(0, '10.180'), (1, '11.090')] -[2023-10-15 14:57:08,443][52518] Saving new best policy, reward=11.090! -[2023-10-15 14:57:09,210][52833] Updated weights for policy 0, policy_version 3720 (0.0008) -[2023-10-15 14:57:09,576][52833] Updated weights for policy 0, policy_version 3730 (0.0008) -[2023-10-15 14:57:09,952][52833] Updated weights for policy 0, policy_version 3740 (0.0007) -[2023-10-15 14:57:10,921][52866] Updated weights for policy 1, policy_version 3750 (0.0011) -[2023-10-15 14:57:11,294][52866] Updated weights for policy 1, policy_version 3760 (0.0008) -[2023-10-15 14:57:11,660][52866] Updated weights for policy 1, policy_version 3770 (0.0011) -[2023-10-15 14:57:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 7700480. Throughput: 0: 1781.4, 1: 1778.1. Samples: 1937348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:57:13,442][51532] Avg episode reward: [(0, '9.890'), (1, '10.600')] -[2023-10-15 14:57:13,654][52833] Updated weights for policy 0, policy_version 3750 (0.0007) -[2023-10-15 14:57:14,019][52833] Updated weights for policy 0, policy_version 3760 (0.0008) -[2023-10-15 14:57:14,398][52833] Updated weights for policy 0, policy_version 3770 (0.0007) -[2023-10-15 14:57:15,571][52866] Updated weights for policy 1, policy_version 3780 (0.0010) -[2023-10-15 14:57:15,937][52866] Updated weights for policy 1, policy_version 3790 (0.0009) -[2023-10-15 14:57:16,291][52866] Updated weights for policy 1, policy_version 3800 (0.0009) -[2023-10-15 14:57:18,288][52833] Updated weights for policy 0, policy_version 3780 (0.0007) -[2023-10-15 14:57:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 7766016. Throughput: 0: 1763.3, 1: 1792.9. Samples: 1947688. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) -[2023-10-15 14:57:18,442][51532] Avg episode reward: [(0, '10.490'), (1, '10.830')] -[2023-10-15 14:57:18,655][52833] Updated weights for policy 0, policy_version 3790 (0.0008) -[2023-10-15 14:57:19,029][52833] Updated weights for policy 0, policy_version 3800 (0.0009) -[2023-10-15 14:57:20,085][52866] Updated weights for policy 1, policy_version 3810 (0.0010) -[2023-10-15 14:57:20,456][52866] Updated weights for policy 1, policy_version 3820 (0.0007) -[2023-10-15 14:57:20,831][52866] Updated weights for policy 1, policy_version 3830 (0.0007) -[2023-10-15 14:57:21,192][52866] Updated weights for policy 1, policy_version 3840 (0.0007) -[2023-10-15 14:57:22,785][52833] Updated weights for policy 0, policy_version 3810 (0.0009) -[2023-10-15 14:57:23,156][52833] Updated weights for policy 0, policy_version 3820 (0.0008) -[2023-10-15 14:57:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 7831552. Throughput: 0: 1774.4, 1: 1786.4. Samples: 1969328. Policy #0 lag: (min: 16.0, avg: 43.8, max: 48.0) -[2023-10-15 14:57:23,442][51532] Avg episode reward: [(0, '10.410'), (1, '10.290')] -[2023-10-15 14:57:23,523][52833] Updated weights for policy 0, policy_version 3830 (0.0008) -[2023-10-15 14:57:23,897][52833] Updated weights for policy 0, policy_version 3840 (0.0010) -[2023-10-15 14:57:24,735][52866] Updated weights for policy 1, policy_version 3850 (0.0009) -[2023-10-15 14:57:25,091][52866] Updated weights for policy 1, policy_version 3860 (0.0008) -[2023-10-15 14:57:25,462][52866] Updated weights for policy 1, policy_version 3870 (0.0009) -[2023-10-15 14:57:27,820][52833] Updated weights for policy 0, policy_version 3850 (0.0007) -[2023-10-15 14:57:28,188][52833] Updated weights for policy 0, policy_version 3860 (0.0007) -[2023-10-15 14:57:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 14106.9). Total num frames: 7897088. Throughput: 0: 1782.4, 1: 1784.8. Samples: 1990938. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 14:57:28,442][51532] Avg episode reward: [(0, '10.710'), (1, '10.540')] -[2023-10-15 14:57:28,560][52833] Updated weights for policy 0, policy_version 3870 (0.0008) -[2023-10-15 14:57:29,428][52866] Updated weights for policy 1, policy_version 3880 (0.0009) -[2023-10-15 14:57:29,814][52866] Updated weights for policy 1, policy_version 3890 (0.0009) -[2023-10-15 14:57:30,183][52866] Updated weights for policy 1, policy_version 3900 (0.0009) -[2023-10-15 14:57:32,379][52833] Updated weights for policy 0, policy_version 3880 (0.0008) -[2023-10-15 14:57:32,756][52833] Updated weights for policy 0, policy_version 3890 (0.0008) -[2023-10-15 14:57:33,124][52833] Updated weights for policy 0, policy_version 3900 (0.0010) -[2023-10-15 14:57:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 7995392. Throughput: 0: 1769.0, 1: 1783.4. Samples: 2001092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:57:33,442][51532] Avg episode reward: [(0, '10.100'), (1, '11.640')] -[2023-10-15 14:57:33,443][52518] Saving new best policy, reward=11.640! -[2023-10-15 14:57:33,908][52866] Updated weights for policy 1, policy_version 3910 (0.0010) -[2023-10-15 14:57:34,276][52866] Updated weights for policy 1, policy_version 3920 (0.0007) -[2023-10-15 14:57:34,656][52866] Updated weights for policy 1, policy_version 3930 (0.0008) -[2023-10-15 14:57:36,897][52833] Updated weights for policy 0, policy_version 3910 (0.0009) -[2023-10-15 14:57:37,269][52833] Updated weights for policy 0, policy_version 3920 (0.0008) -[2023-10-15 14:57:37,637][52833] Updated weights for policy 0, policy_version 3930 (0.0010) -[2023-10-15 14:57:38,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 8060928. Throughput: 0: 1786.8, 1: 1779.8. Samples: 2022792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:57:38,442][51532] Avg episode reward: [(0, '9.850'), (1, '11.250')] -[2023-10-15 14:57:38,489][52866] Updated weights for policy 1, policy_version 3940 (0.0009) -[2023-10-15 14:57:38,865][52866] Updated weights for policy 1, policy_version 3950 (0.0010) -[2023-10-15 14:57:39,240][52866] Updated weights for policy 1, policy_version 3960 (0.0007) -[2023-10-15 14:57:41,511][52833] Updated weights for policy 0, policy_version 3940 (0.0008) -[2023-10-15 14:57:41,872][52833] Updated weights for policy 0, policy_version 3950 (0.0008) -[2023-10-15 14:57:42,244][52833] Updated weights for policy 0, policy_version 3960 (0.0007) -[2023-10-15 14:57:42,933][52866] Updated weights for policy 1, policy_version 3970 (0.0008) -[2023-10-15 14:57:43,308][52866] Updated weights for policy 1, policy_version 3980 (0.0009) -[2023-10-15 14:57:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 8126464. Throughput: 0: 1754.2, 1: 1799.1. Samples: 2043470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:57:43,441][51532] Avg episode reward: [(0, '10.020'), (1, '11.710')] -[2023-10-15 14:57:43,449][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000003968_4063232.pth... -[2023-10-15 14:57:43,481][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000002304_2359296.pth -[2023-10-15 14:57:43,674][52866] Updated weights for policy 1, policy_version 3990 (0.0007) -[2023-10-15 14:57:44,036][52866] Updated weights for policy 1, policy_version 4000 (0.0009) -[2023-10-15 14:57:44,036][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000004000_4096000.pth... -[2023-10-15 14:57:44,075][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000002336_2392064.pth -[2023-10-15 14:57:44,080][52518] Saving new best policy, reward=11.710! -[2023-10-15 14:57:46,127][52833] Updated weights for policy 0, policy_version 3970 (0.0008) -[2023-10-15 14:57:46,506][52833] Updated weights for policy 0, policy_version 3980 (0.0009) -[2023-10-15 14:57:46,881][52833] Updated weights for policy 0, policy_version 3990 (0.0010) -[2023-10-15 14:57:47,254][52833] Updated weights for policy 0, policy_version 4000 (0.0008) -[2023-10-15 14:57:47,888][52866] Updated weights for policy 1, policy_version 4010 (0.0010) -[2023-10-15 14:57:48,261][52866] Updated weights for policy 1, policy_version 4020 (0.0007) -[2023-10-15 14:57:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 8192000. Throughput: 0: 1787.4, 1: 1779.5. Samples: 2054614. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:57:48,441][51532] Avg episode reward: [(0, '9.980'), (1, '11.990')] -[2023-10-15 14:57:48,622][52866] Updated weights for policy 1, policy_version 4030 (0.0008) -[2023-10-15 14:57:48,698][52518] Saving new best policy, reward=11.990! -[2023-10-15 14:57:51,078][52833] Updated weights for policy 0, policy_version 4010 (0.0009) -[2023-10-15 14:57:51,446][52833] Updated weights for policy 0, policy_version 4020 (0.0010) -[2023-10-15 14:57:51,824][52833] Updated weights for policy 0, policy_version 4030 (0.0007) -[2023-10-15 14:57:52,418][52866] Updated weights for policy 1, policy_version 4040 (0.0008) -[2023-10-15 14:57:52,783][52866] Updated weights for policy 1, policy_version 4050 (0.0009) -[2023-10-15 14:57:53,153][52866] Updated weights for policy 1, policy_version 4060 (0.0010) -[2023-10-15 14:57:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14106.9). Total num frames: 8290304. Throughput: 0: 1756.0, 1: 1797.7. Samples: 2075498. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 14:57:53,441][51532] Avg episode reward: [(0, '10.830'), (1, '11.630')] -[2023-10-15 14:57:53,442][52410] Saving new best policy, reward=10.830! -[2023-10-15 14:57:55,629][52833] Updated weights for policy 0, policy_version 4040 (0.0009) -[2023-10-15 14:57:56,000][52833] Updated weights for policy 0, policy_version 4050 (0.0008) -[2023-10-15 14:57:56,360][52833] Updated weights for policy 0, policy_version 4060 (0.0008) -[2023-10-15 14:57:57,044][52866] Updated weights for policy 1, policy_version 4070 (0.0009) -[2023-10-15 14:57:57,423][52866] Updated weights for policy 1, policy_version 4080 (0.0011) -[2023-10-15 14:57:57,797][52866] Updated weights for policy 1, policy_version 4090 (0.0010) -[2023-10-15 14:57:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14106.9). Total num frames: 8355840. Throughput: 0: 1755.4, 1: 1775.3. Samples: 2096230. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 14:57:58,442][51532] Avg episode reward: [(0, '10.340'), (1, '10.790')] -[2023-10-15 14:58:00,162][52833] Updated weights for policy 0, policy_version 4070 (0.0008) -[2023-10-15 14:58:00,535][52833] Updated weights for policy 0, policy_version 4080 (0.0007) -[2023-10-15 14:58:00,906][52833] Updated weights for policy 0, policy_version 4090 (0.0007) -[2023-10-15 14:58:01,575][52866] Updated weights for policy 1, policy_version 4100 (0.0009) -[2023-10-15 14:58:01,940][52866] Updated weights for policy 1, policy_version 4110 (0.0009) -[2023-10-15 14:58:02,312][52866] Updated weights for policy 1, policy_version 4120 (0.0008) -[2023-10-15 14:58:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 8421376. Throughput: 0: 1766.3, 1: 1792.0. Samples: 2107812. Policy #0 lag: (min: 2.0, avg: 10.9, max: 34.0) -[2023-10-15 14:58:03,442][51532] Avg episode reward: [(0, '10.720'), (1, '10.470')] -[2023-10-15 14:58:04,684][52833] Updated weights for policy 0, policy_version 4100 (0.0008) -[2023-10-15 14:58:05,044][52833] Updated weights for policy 0, policy_version 4110 (0.0008) -[2023-10-15 14:58:05,421][52833] Updated weights for policy 0, policy_version 4120 (0.0008) -[2023-10-15 14:58:05,951][52866] Updated weights for policy 1, policy_version 4130 (0.0007) -[2023-10-15 14:58:06,324][52866] Updated weights for policy 1, policy_version 4140 (0.0009) -[2023-10-15 14:58:06,680][52866] Updated weights for policy 1, policy_version 4150 (0.0009) -[2023-10-15 14:58:07,049][52866] Updated weights for policy 1, policy_version 4160 (0.0010) -[2023-10-15 14:58:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 8486912. Throughput: 0: 1753.1, 1: 1778.1. Samples: 2128234. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-15 14:58:08,442][51532] Avg episode reward: [(0, '10.180'), (1, '10.800')] -[2023-10-15 14:58:09,281][52833] Updated weights for policy 0, policy_version 4130 (0.0008) -[2023-10-15 14:58:09,650][52833] Updated weights for policy 0, policy_version 4140 (0.0010) -[2023-10-15 14:58:10,020][52833] Updated weights for policy 0, policy_version 4150 (0.0011) -[2023-10-15 14:58:10,390][52833] Updated weights for policy 0, policy_version 4160 (0.0010) -[2023-10-15 14:58:10,707][52866] Updated weights for policy 1, policy_version 4170 (0.0007) -[2023-10-15 14:58:11,062][52866] Updated weights for policy 1, policy_version 4180 (0.0007) -[2023-10-15 14:58:11,433][52866] Updated weights for policy 1, policy_version 4190 (0.0009) -[2023-10-15 14:58:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 8552448. Throughput: 0: 1761.0, 1: 1782.9. Samples: 2150414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:58:13,442][51532] Avg episode reward: [(0, '10.660'), (1, '10.780')] -[2023-10-15 14:58:14,295][52833] Updated weights for policy 0, policy_version 4170 (0.0007) -[2023-10-15 14:58:14,664][52833] Updated weights for policy 0, policy_version 4180 (0.0007) -[2023-10-15 14:58:15,035][52833] Updated weights for policy 0, policy_version 4190 (0.0007) -[2023-10-15 14:58:15,278][52866] Updated weights for policy 1, policy_version 4200 (0.0009) -[2023-10-15 14:58:15,653][52866] Updated weights for policy 1, policy_version 4210 (0.0010) -[2023-10-15 14:58:16,009][52866] Updated weights for policy 1, policy_version 4220 (0.0009) -[2023-10-15 14:58:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 8617984. Throughput: 0: 1754.6, 1: 1788.7. Samples: 2160538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:58:18,441][51532] Avg episode reward: [(0, '10.730'), (1, '11.390')] -[2023-10-15 14:58:18,787][52833] Updated weights for policy 0, policy_version 4200 (0.0007) -[2023-10-15 14:58:19,156][52833] Updated weights for policy 0, policy_version 4210 (0.0007) -[2023-10-15 14:58:19,525][52833] Updated weights for policy 0, policy_version 4220 (0.0008) -[2023-10-15 14:58:19,635][52866] Updated weights for policy 1, policy_version 4230 (0.0008) -[2023-10-15 14:58:19,992][52866] Updated weights for policy 1, policy_version 4240 (0.0008) -[2023-10-15 14:58:20,353][52866] Updated weights for policy 1, policy_version 4250 (0.0008) -[2023-10-15 14:58:23,161][52833] Updated weights for policy 0, policy_version 4230 (0.0009) -[2023-10-15 14:58:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 8683520. Throughput: 0: 1766.8, 1: 1788.0. Samples: 2182754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:58:23,441][51532] Avg episode reward: [(0, '10.740'), (1, '10.870')] -[2023-10-15 14:58:23,527][52833] Updated weights for policy 0, policy_version 4240 (0.0009) -[2023-10-15 14:58:23,901][52833] Updated weights for policy 0, policy_version 4250 (0.0008) -[2023-10-15 14:58:24,159][52866] Updated weights for policy 1, policy_version 4260 (0.0009) -[2023-10-15 14:58:24,516][52866] Updated weights for policy 1, policy_version 4270 (0.0009) -[2023-10-15 14:58:24,879][52866] Updated weights for policy 1, policy_version 4280 (0.0011) -[2023-10-15 14:58:27,618][52833] Updated weights for policy 0, policy_version 4260 (0.0008) -[2023-10-15 14:58:27,989][52833] Updated weights for policy 0, policy_version 4270 (0.0011) -[2023-10-15 14:58:28,364][52833] Updated weights for policy 0, policy_version 4280 (0.0010) -[2023-10-15 14:58:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 8749056. Throughput: 0: 1789.2, 1: 1786.4. Samples: 2204372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:58:28,441][51532] Avg episode reward: [(0, '11.600'), (1, '11.720')] -[2023-10-15 14:58:28,654][52410] Saving new best policy, reward=11.600! -[2023-10-15 14:58:28,828][52866] Updated weights for policy 1, policy_version 4290 (0.0010) -[2023-10-15 14:58:29,194][52866] Updated weights for policy 1, policy_version 4300 (0.0011) -[2023-10-15 14:58:29,557][52866] Updated weights for policy 1, policy_version 4310 (0.0010) -[2023-10-15 14:58:29,923][52866] Updated weights for policy 1, policy_version 4320 (0.0009) -[2023-10-15 14:58:32,091][52833] Updated weights for policy 0, policy_version 4290 (0.0008) -[2023-10-15 14:58:32,465][52833] Updated weights for policy 0, policy_version 4300 (0.0009) -[2023-10-15 14:58:32,836][52833] Updated weights for policy 0, policy_version 4310 (0.0009) -[2023-10-15 14:58:33,207][52833] Updated weights for policy 0, policy_version 4320 (0.0007) -[2023-10-15 14:58:33,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 8847360. Throughput: 0: 1769.8, 1: 1782.3. Samples: 2214456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:58:33,442][51532] Avg episode reward: [(0, '10.470'), (1, '11.730')] -[2023-10-15 14:58:33,794][52866] Updated weights for policy 1, policy_version 4330 (0.0008) -[2023-10-15 14:58:34,157][52866] Updated weights for policy 1, policy_version 4340 (0.0007) -[2023-10-15 14:58:34,524][52866] Updated weights for policy 1, policy_version 4350 (0.0007) -[2023-10-15 14:58:36,927][52833] Updated weights for policy 0, policy_version 4330 (0.0009) -[2023-10-15 14:58:37,296][52833] Updated weights for policy 0, policy_version 4340 (0.0009) -[2023-10-15 14:58:37,664][52833] Updated weights for policy 0, policy_version 4350 (0.0010) -[2023-10-15 14:58:38,214][52866] Updated weights for policy 1, policy_version 4360 (0.0009) -[2023-10-15 14:58:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 8912896. Throughput: 0: 1798.5, 1: 1779.7. Samples: 2236518. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:58:38,442][51532] Avg episode reward: [(0, '10.790'), (1, '10.860')] -[2023-10-15 14:58:38,579][52866] Updated weights for policy 1, policy_version 4370 (0.0010) -[2023-10-15 14:58:38,940][52866] Updated weights for policy 1, policy_version 4380 (0.0010) -[2023-10-15 14:58:41,345][52833] Updated weights for policy 0, policy_version 4360 (0.0008) -[2023-10-15 14:58:41,713][52833] Updated weights for policy 0, policy_version 4370 (0.0007) -[2023-10-15 14:58:42,094][52833] Updated weights for policy 0, policy_version 4380 (0.0009) -[2023-10-15 14:58:42,847][52866] Updated weights for policy 1, policy_version 4390 (0.0009) -[2023-10-15 14:58:43,207][52866] Updated weights for policy 1, policy_version 4400 (0.0008) -[2023-10-15 14:58:43,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 8978432. Throughput: 0: 1778.8, 1: 1802.7. Samples: 2257396. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:58:43,441][51532] Avg episode reward: [(0, '10.460'), (1, '11.720')] -[2023-10-15 14:58:43,566][52866] Updated weights for policy 1, policy_version 4410 (0.0008) -[2023-10-15 14:58:45,913][52833] Updated weights for policy 0, policy_version 4390 (0.0009) -[2023-10-15 14:58:46,279][52833] Updated weights for policy 0, policy_version 4400 (0.0008) -[2023-10-15 14:58:46,652][52833] Updated weights for policy 0, policy_version 4410 (0.0007) -[2023-10-15 14:58:47,400][52866] Updated weights for policy 1, policy_version 4420 (0.0008) -[2023-10-15 14:58:47,767][52866] Updated weights for policy 1, policy_version 4430 (0.0007) -[2023-10-15 14:58:48,134][52866] Updated weights for policy 1, policy_version 4440 (0.0008) -[2023-10-15 14:58:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 9076736. Throughput: 0: 1796.5, 1: 1779.5. Samples: 2268734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:58:48,441][51532] Avg episode reward: [(0, '11.600'), (1, '12.440')] -[2023-10-15 14:58:48,442][52518] Saving new best policy, reward=12.440! -[2023-10-15 14:58:50,475][52833] Updated weights for policy 0, policy_version 4420 (0.0009) -[2023-10-15 14:58:50,856][52833] Updated weights for policy 0, policy_version 4430 (0.0008) -[2023-10-15 14:58:51,220][52833] Updated weights for policy 0, policy_version 4440 (0.0008) -[2023-10-15 14:58:51,909][52866] Updated weights for policy 1, policy_version 4450 (0.0008) -[2023-10-15 14:58:52,278][52866] Updated weights for policy 1, policy_version 4460 (0.0007) -[2023-10-15 14:58:52,653][52866] Updated weights for policy 1, policy_version 4470 (0.0010) -[2023-10-15 14:58:53,016][52866] Updated weights for policy 1, policy_version 4480 (0.0010) -[2023-10-15 14:58:53,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 9142272. Throughput: 0: 1779.6, 1: 1807.1. Samples: 2289638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:58:53,442][51532] Avg episode reward: [(0, '10.290'), (1, '10.820')] -[2023-10-15 14:58:54,962][52833] Updated weights for policy 0, policy_version 4450 (0.0008) -[2023-10-15 14:58:55,329][52833] Updated weights for policy 0, policy_version 4460 (0.0009) -[2023-10-15 14:58:55,704][52833] Updated weights for policy 0, policy_version 4470 (0.0007) -[2023-10-15 14:58:56,073][52833] Updated weights for policy 0, policy_version 4480 (0.0007) -[2023-10-15 14:58:56,941][52866] Updated weights for policy 1, policy_version 4490 (0.0008) -[2023-10-15 14:58:57,304][52866] Updated weights for policy 1, policy_version 4500 (0.0008) -[2023-10-15 14:58:57,670][52866] Updated weights for policy 1, policy_version 4510 (0.0007) -[2023-10-15 14:58:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 9207808. Throughput: 0: 1790.7, 1: 1771.1. Samples: 2310694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:58:58,442][51532] Avg episode reward: [(0, '11.170'), (1, '11.100')] -[2023-10-15 14:59:00,089][52833] Updated weights for policy 0, policy_version 4490 (0.0007) -[2023-10-15 14:59:00,460][52833] Updated weights for policy 0, policy_version 4500 (0.0007) -[2023-10-15 14:59:00,823][52833] Updated weights for policy 0, policy_version 4510 (0.0010) -[2023-10-15 14:59:01,359][52866] Updated weights for policy 1, policy_version 4520 (0.0011) -[2023-10-15 14:59:01,732][52866] Updated weights for policy 1, policy_version 4530 (0.0010) -[2023-10-15 14:59:02,098][52866] Updated weights for policy 1, policy_version 4540 (0.0008) -[2023-10-15 14:59:03,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 9273344. Throughput: 0: 1781.6, 1: 1796.7. Samples: 2321564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:59:03,441][51532] Avg episode reward: [(0, '10.750'), (1, '10.980')] -[2023-10-15 14:59:04,606][52833] Updated weights for policy 0, policy_version 4520 (0.0007) -[2023-10-15 14:59:04,977][52833] Updated weights for policy 0, policy_version 4530 (0.0009) -[2023-10-15 14:59:05,355][52833] Updated weights for policy 0, policy_version 4540 (0.0007) -[2023-10-15 14:59:05,894][52866] Updated weights for policy 1, policy_version 4550 (0.0011) -[2023-10-15 14:59:06,266][52866] Updated weights for policy 1, policy_version 4560 (0.0008) -[2023-10-15 14:59:06,626][52866] Updated weights for policy 1, policy_version 4570 (0.0008) -[2023-10-15 14:59:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 9338880. Throughput: 0: 1773.6, 1: 1766.6. Samples: 2342064. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:59:08,442][51532] Avg episode reward: [(0, '11.720'), (1, '11.820')] -[2023-10-15 14:59:08,443][52410] Saving new best policy, reward=11.720! -[2023-10-15 14:59:09,180][52833] Updated weights for policy 0, policy_version 4550 (0.0008) -[2023-10-15 14:59:09,546][52833] Updated weights for policy 0, policy_version 4560 (0.0008) -[2023-10-15 14:59:09,911][52833] Updated weights for policy 0, policy_version 4570 (0.0009) -[2023-10-15 14:59:10,421][52866] Updated weights for policy 1, policy_version 4580 (0.0007) -[2023-10-15 14:59:10,794][52866] Updated weights for policy 1, policy_version 4590 (0.0007) -[2023-10-15 14:59:11,161][52866] Updated weights for policy 1, policy_version 4600 (0.0008) -[2023-10-15 14:59:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 9404416. Throughput: 0: 1784.2, 1: 1775.7. Samples: 2364568. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 14:59:13,442][51532] Avg episode reward: [(0, '11.680'), (1, '11.170')] -[2023-10-15 14:59:13,750][52833] Updated weights for policy 0, policy_version 4580 (0.0010) -[2023-10-15 14:59:14,122][52833] Updated weights for policy 0, policy_version 4590 (0.0011) -[2023-10-15 14:59:14,494][52833] Updated weights for policy 0, policy_version 4600 (0.0008) -[2023-10-15 14:59:14,786][52866] Updated weights for policy 1, policy_version 4610 (0.0007) -[2023-10-15 14:59:15,148][52866] Updated weights for policy 1, policy_version 4620 (0.0007) -[2023-10-15 14:59:15,511][52866] Updated weights for policy 1, policy_version 4630 (0.0007) -[2023-10-15 14:59:15,879][52866] Updated weights for policy 1, policy_version 4640 (0.0010) -[2023-10-15 14:59:18,271][52833] Updated weights for policy 0, policy_version 4610 (0.0007) -[2023-10-15 14:59:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 9469952. Throughput: 0: 1775.7, 1: 1780.5. Samples: 2374482. Policy #0 lag: (min: 10.0, avg: 10.0, max: 12.0) -[2023-10-15 14:59:18,442][51532] Avg episode reward: [(0, '11.080'), (1, '11.570')] -[2023-10-15 14:59:18,634][52833] Updated weights for policy 0, policy_version 4620 (0.0007) -[2023-10-15 14:59:19,003][52833] Updated weights for policy 0, policy_version 4630 (0.0008) -[2023-10-15 14:59:19,375][52833] Updated weights for policy 0, policy_version 4640 (0.0007) -[2023-10-15 14:59:19,715][52866] Updated weights for policy 1, policy_version 4650 (0.0010) -[2023-10-15 14:59:20,087][52866] Updated weights for policy 1, policy_version 4660 (0.0009) -[2023-10-15 14:59:20,458][52866] Updated weights for policy 1, policy_version 4670 (0.0011) -[2023-10-15 14:59:23,207][52833] Updated weights for policy 0, policy_version 4650 (0.0007) -[2023-10-15 14:59:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 9535488. Throughput: 0: 1775.4, 1: 1778.5. Samples: 2396442. Policy #0 lag: (min: 10.0, avg: 10.0, max: 12.0) -[2023-10-15 14:59:23,442][51532] Avg episode reward: [(0, '11.010'), (1, '11.250')] -[2023-10-15 14:59:23,578][52833] Updated weights for policy 0, policy_version 4660 (0.0007) -[2023-10-15 14:59:23,943][52833] Updated weights for policy 0, policy_version 4670 (0.0009) -[2023-10-15 14:59:24,121][52866] Updated weights for policy 1, policy_version 4680 (0.0008) -[2023-10-15 14:59:24,484][52866] Updated weights for policy 1, policy_version 4690 (0.0007) -[2023-10-15 14:59:24,857][52866] Updated weights for policy 1, policy_version 4700 (0.0010) -[2023-10-15 14:59:27,700][52833] Updated weights for policy 0, policy_version 4680 (0.0009) -[2023-10-15 14:59:28,066][52833] Updated weights for policy 0, policy_version 4690 (0.0010) -[2023-10-15 14:59:28,441][52833] Updated weights for policy 0, policy_version 4700 (0.0008) -[2023-10-15 14:59:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 9601024. Throughput: 0: 1782.7, 1: 1791.8. Samples: 2418246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 14:59:28,442][51532] Avg episode reward: [(0, '10.360'), (1, '12.010')] -[2023-10-15 14:59:28,654][52866] Updated weights for policy 1, policy_version 4710 (0.0008) -[2023-10-15 14:59:29,017][52866] Updated weights for policy 1, policy_version 4720 (0.0008) -[2023-10-15 14:59:29,391][52866] Updated weights for policy 1, policy_version 4730 (0.0007) -[2023-10-15 14:59:32,196][52833] Updated weights for policy 0, policy_version 4710 (0.0010) -[2023-10-15 14:59:32,573][52833] Updated weights for policy 0, policy_version 4720 (0.0011) -[2023-10-15 14:59:32,941][52833] Updated weights for policy 0, policy_version 4730 (0.0009) -[2023-10-15 14:59:33,177][52866] Updated weights for policy 1, policy_version 4740 (0.0008) -[2023-10-15 14:59:33,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 9699328. Throughput: 0: 1768.7, 1: 1781.9. Samples: 2428512. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 14:59:33,442][51532] Avg episode reward: [(0, '10.130'), (1, '12.190')] -[2023-10-15 14:59:33,544][52866] Updated weights for policy 1, policy_version 4750 (0.0008) -[2023-10-15 14:59:33,915][52866] Updated weights for policy 1, policy_version 4760 (0.0008) -[2023-10-15 14:59:36,785][52833] Updated weights for policy 0, policy_version 4740 (0.0007) -[2023-10-15 14:59:37,151][52833] Updated weights for policy 0, policy_version 4750 (0.0007) -[2023-10-15 14:59:37,521][52833] Updated weights for policy 0, policy_version 4760 (0.0008) -[2023-10-15 14:59:37,816][52866] Updated weights for policy 1, policy_version 4770 (0.0008) -[2023-10-15 14:59:38,191][52866] Updated weights for policy 1, policy_version 4780 (0.0008) -[2023-10-15 14:59:38,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 9764864. Throughput: 0: 1788.6, 1: 1783.6. Samples: 2450388. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 14:59:38,442][51532] Avg episode reward: [(0, '10.070'), (1, '12.760')] -[2023-10-15 14:59:38,562][52866] Updated weights for policy 1, policy_version 4790 (0.0009) -[2023-10-15 14:59:38,927][52518] Saving new best policy, reward=12.760! -[2023-10-15 14:59:38,931][52866] Updated weights for policy 1, policy_version 4800 (0.0009) -[2023-10-15 14:59:41,250][52833] Updated weights for policy 0, policy_version 4770 (0.0007) -[2023-10-15 14:59:41,620][52833] Updated weights for policy 0, policy_version 4780 (0.0009) -[2023-10-15 14:59:41,989][52833] Updated weights for policy 0, policy_version 4790 (0.0008) -[2023-10-15 14:59:42,359][52833] Updated weights for policy 0, policy_version 4800 (0.0008) -[2023-10-15 14:59:42,689][52866] Updated weights for policy 1, policy_version 4810 (0.0008) -[2023-10-15 14:59:43,059][52866] Updated weights for policy 1, policy_version 4820 (0.0008) -[2023-10-15 14:59:43,424][52866] Updated weights for policy 1, policy_version 4830 (0.0010) -[2023-10-15 14:59:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 9830400. Throughput: 0: 1755.3, 1: 1798.8. Samples: 2470630. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-15 14:59:43,441][51532] Avg episode reward: [(0, '10.960'), (1, '12.270')] -[2023-10-15 14:59:43,449][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000004800_4915200.pth... -[2023-10-15 14:59:43,479][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000003136_3211264.pth -[2023-10-15 14:59:43,498][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000004832_4947968.pth... -[2023-10-15 14:59:43,527][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000003168_3244032.pth -[2023-10-15 14:59:46,215][52833] Updated weights for policy 0, policy_version 4810 (0.0007) -[2023-10-15 14:59:46,589][52833] Updated weights for policy 0, policy_version 4820 (0.0009) -[2023-10-15 14:59:46,967][52833] Updated weights for policy 0, policy_version 4830 (0.0009) -[2023-10-15 14:59:47,253][52866] Updated weights for policy 1, policy_version 4840 (0.0008) -[2023-10-15 14:59:47,625][52866] Updated weights for policy 1, policy_version 4850 (0.0008) -[2023-10-15 14:59:47,994][52866] Updated weights for policy 1, policy_version 4860 (0.0009) -[2023-10-15 14:59:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 9928704. Throughput: 0: 1788.6, 1: 1784.5. Samples: 2482352. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-15 14:59:48,442][51532] Avg episode reward: [(0, '10.260'), (1, '12.580')] -[2023-10-15 14:59:50,910][52833] Updated weights for policy 0, policy_version 4840 (0.0007) -[2023-10-15 14:59:51,283][52833] Updated weights for policy 0, policy_version 4850 (0.0008) -[2023-10-15 14:59:51,645][52833] Updated weights for policy 0, policy_version 4860 (0.0009) -[2023-10-15 14:59:51,797][52866] Updated weights for policy 1, policy_version 4870 (0.0008) -[2023-10-15 14:59:52,164][52866] Updated weights for policy 1, policy_version 4880 (0.0007) -[2023-10-15 14:59:52,534][52866] Updated weights for policy 1, policy_version 4890 (0.0009) -[2023-10-15 14:59:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 9994240. Throughput: 0: 1760.3, 1: 1797.3. Samples: 2502154. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-15 14:59:53,442][51532] Avg episode reward: [(0, '10.570'), (1, '12.460')] -[2023-10-15 14:59:55,407][52833] Updated weights for policy 0, policy_version 4870 (0.0009) -[2023-10-15 14:59:55,793][52833] Updated weights for policy 0, policy_version 4880 (0.0009) -[2023-10-15 14:59:56,146][52866] Updated weights for policy 1, policy_version 4900 (0.0007) -[2023-10-15 14:59:56,165][52833] Updated weights for policy 0, policy_version 4890 (0.0009) -[2023-10-15 14:59:56,510][52866] Updated weights for policy 1, policy_version 4910 (0.0008) -[2023-10-15 14:59:56,879][52866] Updated weights for policy 1, policy_version 4920 (0.0008) -[2023-10-15 14:59:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 10059776. Throughput: 0: 1755.3, 1: 1775.1. Samples: 2523438. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-15 14:59:58,442][51532] Avg episode reward: [(0, '10.440'), (1, '13.130')] -[2023-10-15 14:59:58,453][52518] Saving new best policy, reward=13.130! -[2023-10-15 14:59:59,946][52833] Updated weights for policy 0, policy_version 4900 (0.0009) -[2023-10-15 15:00:00,317][52833] Updated weights for policy 0, policy_version 4910 (0.0008) -[2023-10-15 15:00:00,635][52866] Updated weights for policy 1, policy_version 4930 (0.0009) -[2023-10-15 15:00:00,680][52833] Updated weights for policy 0, policy_version 4920 (0.0007) -[2023-10-15 15:00:00,991][52866] Updated weights for policy 1, policy_version 4940 (0.0007) -[2023-10-15 15:00:01,361][52866] Updated weights for policy 1, policy_version 4950 (0.0011) -[2023-10-15 15:00:01,740][52866] Updated weights for policy 1, policy_version 4960 (0.0008) -[2023-10-15 15:00:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 10125312. Throughput: 0: 1757.3, 1: 1795.8. Samples: 2534372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:00:03,441][51532] Avg episode reward: [(0, '11.230'), (1, '13.290')] -[2023-10-15 15:00:03,442][52518] Saving new best policy, reward=13.290! -[2023-10-15 15:00:04,600][52833] Updated weights for policy 0, policy_version 4930 (0.0008) -[2023-10-15 15:00:04,963][52833] Updated weights for policy 0, policy_version 4940 (0.0009) -[2023-10-15 15:00:05,328][52833] Updated weights for policy 0, policy_version 4950 (0.0007) -[2023-10-15 15:00:05,508][52866] Updated weights for policy 1, policy_version 4970 (0.0007) -[2023-10-15 15:00:05,698][52833] Updated weights for policy 0, policy_version 4960 (0.0007) -[2023-10-15 15:00:05,884][52866] Updated weights for policy 1, policy_version 4980 (0.0007) -[2023-10-15 15:00:06,254][52866] Updated weights for policy 1, policy_version 4990 (0.0008) -[2023-10-15 15:00:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 10190848. Throughput: 0: 1759.7, 1: 1773.9. Samples: 2555456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:00:08,442][51532] Avg episode reward: [(0, '10.970'), (1, '12.760')] -[2023-10-15 15:00:09,402][52833] Updated weights for policy 0, policy_version 4970 (0.0010) -[2023-10-15 15:00:09,774][52833] Updated weights for policy 0, policy_version 4980 (0.0007) -[2023-10-15 15:00:10,134][52833] Updated weights for policy 0, policy_version 4990 (0.0009) -[2023-10-15 15:00:10,137][52866] Updated weights for policy 1, policy_version 5000 (0.0007) -[2023-10-15 15:00:10,503][52866] Updated weights for policy 1, policy_version 5010 (0.0007) -[2023-10-15 15:00:10,876][52866] Updated weights for policy 1, policy_version 5020 (0.0009) -[2023-10-15 15:00:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 10256384. Throughput: 0: 1778.4, 1: 1776.4. Samples: 2578210. Policy #0 lag: (min: 14.0, avg: 38.4, max: 40.0) -[2023-10-15 15:00:13,442][51532] Avg episode reward: [(0, '11.040'), (1, '12.190')] -[2023-10-15 15:00:13,820][52833] Updated weights for policy 0, policy_version 5000 (0.0008) -[2023-10-15 15:00:14,190][52833] Updated weights for policy 0, policy_version 5010 (0.0008) -[2023-10-15 15:00:14,427][52866] Updated weights for policy 1, policy_version 5030 (0.0007) -[2023-10-15 15:00:14,562][52833] Updated weights for policy 0, policy_version 5020 (0.0007) -[2023-10-15 15:00:14,797][52866] Updated weights for policy 1, policy_version 5040 (0.0007) -[2023-10-15 15:00:15,173][52866] Updated weights for policy 1, policy_version 5050 (0.0007) -[2023-10-15 15:00:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 10321920. Throughput: 0: 1762.3, 1: 1780.1. Samples: 2587918. Policy #0 lag: (min: 14.0, avg: 38.4, max: 40.0) -[2023-10-15 15:00:18,441][51532] Avg episode reward: [(0, '10.140'), (1, '12.330')] -[2023-10-15 15:00:18,495][52833] Updated weights for policy 0, policy_version 5030 (0.0008) -[2023-10-15 15:00:18,857][52833] Updated weights for policy 0, policy_version 5040 (0.0009) -[2023-10-15 15:00:19,093][52866] Updated weights for policy 1, policy_version 5060 (0.0008) -[2023-10-15 15:00:19,229][52833] Updated weights for policy 0, policy_version 5050 (0.0009) -[2023-10-15 15:00:19,455][52866] Updated weights for policy 1, policy_version 5070 (0.0008) -[2023-10-15 15:00:19,825][52866] Updated weights for policy 1, policy_version 5080 (0.0008) -[2023-10-15 15:00:23,189][52833] Updated weights for policy 0, policy_version 5060 (0.0007) -[2023-10-15 15:00:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 10387456. Throughput: 0: 1765.6, 1: 1781.4. Samples: 2610004. Policy #0 lag: (min: 27.0, avg: 29.6, max: 56.0) -[2023-10-15 15:00:23,441][51532] Avg episode reward: [(0, '10.510'), (1, '13.160')] -[2023-10-15 15:00:23,536][52866] Updated weights for policy 1, policy_version 5090 (0.0008) -[2023-10-15 15:00:23,554][52833] Updated weights for policy 0, policy_version 5070 (0.0007) -[2023-10-15 15:00:23,896][52866] Updated weights for policy 1, policy_version 5100 (0.0010) -[2023-10-15 15:00:23,928][52833] Updated weights for policy 0, policy_version 5080 (0.0007) -[2023-10-15 15:00:24,274][52866] Updated weights for policy 1, policy_version 5110 (0.0008) -[2023-10-15 15:00:24,643][52866] Updated weights for policy 1, policy_version 5120 (0.0010) -[2023-10-15 15:00:27,804][52833] Updated weights for policy 0, policy_version 5090 (0.0007) -[2023-10-15 15:00:28,174][52833] Updated weights for policy 0, policy_version 5100 (0.0008) -[2023-10-15 15:00:28,438][52866] Updated weights for policy 1, policy_version 5130 (0.0007) -[2023-10-15 15:00:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 10452992. Throughput: 0: 1786.1, 1: 1800.6. Samples: 2632032. Policy #0 lag: (min: 27.0, avg: 29.6, max: 56.0) -[2023-10-15 15:00:28,442][51532] Avg episode reward: [(0, '10.680'), (1, '14.130')] -[2023-10-15 15:00:28,543][52833] Updated weights for policy 0, policy_version 5110 (0.0008) -[2023-10-15 15:00:28,797][52866] Updated weights for policy 1, policy_version 5140 (0.0010) -[2023-10-15 15:00:28,912][52833] Updated weights for policy 0, policy_version 5120 (0.0008) -[2023-10-15 15:00:29,172][52866] Updated weights for policy 1, policy_version 5150 (0.0008) -[2023-10-15 15:00:29,237][52518] Saving new best policy, reward=14.130! -[2023-10-15 15:00:32,719][52833] Updated weights for policy 0, policy_version 5130 (0.0009) -[2023-10-15 15:00:32,953][52866] Updated weights for policy 1, policy_version 5160 (0.0007) -[2023-10-15 15:00:33,093][52833] Updated weights for policy 0, policy_version 5140 (0.0008) -[2023-10-15 15:00:33,325][52866] Updated weights for policy 1, policy_version 5170 (0.0008) -[2023-10-15 15:00:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 14106.9). Total num frames: 10518528. Throughput: 0: 1757.9, 1: 1782.8. Samples: 2641680. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-15 15:00:33,441][51532] Avg episode reward: [(0, '10.620'), (1, '12.850')] -[2023-10-15 15:00:33,458][52833] Updated weights for policy 0, policy_version 5150 (0.0008) -[2023-10-15 15:00:33,692][52866] Updated weights for policy 1, policy_version 5180 (0.0008) -[2023-10-15 15:00:37,371][52833] Updated weights for policy 0, policy_version 5160 (0.0007) -[2023-10-15 15:00:37,540][52866] Updated weights for policy 1, policy_version 5190 (0.0009) -[2023-10-15 15:00:37,738][52833] Updated weights for policy 0, policy_version 5170 (0.0009) -[2023-10-15 15:00:37,909][52866] Updated weights for policy 1, policy_version 5200 (0.0008) -[2023-10-15 15:00:38,106][52833] Updated weights for policy 0, policy_version 5180 (0.0009) -[2023-10-15 15:00:38,278][52866] Updated weights for policy 1, policy_version 5210 (0.0007) -[2023-10-15 15:00:38,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 10616832. Throughput: 0: 1782.4, 1: 1799.2. Samples: 2663330. Policy #0 lag: (min: 26.0, avg: 27.7, max: 54.0) -[2023-10-15 15:00:38,442][51532] Avg episode reward: [(0, '10.680'), (1, '12.690')] -[2023-10-15 15:00:42,101][52866] Updated weights for policy 1, policy_version 5220 (0.0008) -[2023-10-15 15:00:42,119][52833] Updated weights for policy 0, policy_version 5190 (0.0008) -[2023-10-15 15:00:42,466][52866] Updated weights for policy 1, policy_version 5230 (0.0009) -[2023-10-15 15:00:42,508][52833] Updated weights for policy 0, policy_version 5200 (0.0009) -[2023-10-15 15:00:42,837][52866] Updated weights for policy 1, policy_version 5240 (0.0008) -[2023-10-15 15:00:42,875][52833] Updated weights for policy 0, policy_version 5210 (0.0008) -[2023-10-15 15:00:43,441][51532] Fps is (10 sec: 19660.0, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 10715136. Throughput: 0: 1752.8, 1: 1791.5. Samples: 2682932. Policy #0 lag: (min: 26.0, avg: 27.7, max: 54.0) -[2023-10-15 15:00:43,442][51532] Avg episode reward: [(0, '10.770'), (1, '12.400')] -[2023-10-15 15:00:46,587][52833] Updated weights for policy 0, policy_version 5220 (0.0007) -[2023-10-15 15:00:46,597][52866] Updated weights for policy 1, policy_version 5250 (0.0008) -[2023-10-15 15:00:46,962][52866] Updated weights for policy 1, policy_version 5260 (0.0008) -[2023-10-15 15:00:46,964][52833] Updated weights for policy 0, policy_version 5230 (0.0007) -[2023-10-15 15:00:47,332][52866] Updated weights for policy 1, policy_version 5270 (0.0007) -[2023-10-15 15:00:47,340][52833] Updated weights for policy 0, policy_version 5240 (0.0008) -[2023-10-15 15:00:47,689][52866] Updated weights for policy 1, policy_version 5280 (0.0009) -[2023-10-15 15:00:48,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 10780672. Throughput: 0: 1771.9, 1: 1796.8. Samples: 2694962. Policy #0 lag: (min: 1.0, avg: 7.8, max: 33.0) -[2023-10-15 15:00:48,441][51532] Avg episode reward: [(0, '11.060'), (1, '11.880')] -[2023-10-15 15:00:51,175][52833] Updated weights for policy 0, policy_version 5250 (0.0008) -[2023-10-15 15:00:51,540][52833] Updated weights for policy 0, policy_version 5260 (0.0008) -[2023-10-15 15:00:51,585][52866] Updated weights for policy 1, policy_version 5290 (0.0009) -[2023-10-15 15:00:51,907][52833] Updated weights for policy 0, policy_version 5270 (0.0007) -[2023-10-15 15:00:51,956][52866] Updated weights for policy 1, policy_version 5300 (0.0008) -[2023-10-15 15:00:52,265][52833] Updated weights for policy 0, policy_version 5280 (0.0007) -[2023-10-15 15:00:52,330][52866] Updated weights for policy 1, policy_version 5310 (0.0009) -[2023-10-15 15:00:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 10846208. Throughput: 0: 1757.2, 1: 1796.0. Samples: 2715350. Policy #0 lag: (min: 1.0, avg: 7.8, max: 33.0) -[2023-10-15 15:00:53,441][51532] Avg episode reward: [(0, '10.970'), (1, '11.320')] -[2023-10-15 15:00:56,041][52866] Updated weights for policy 1, policy_version 5320 (0.0009) -[2023-10-15 15:00:56,107][52833] Updated weights for policy 0, policy_version 5290 (0.0009) -[2023-10-15 15:00:56,413][52866] Updated weights for policy 1, policy_version 5330 (0.0008) -[2023-10-15 15:00:56,466][52833] Updated weights for policy 0, policy_version 5300 (0.0008) -[2023-10-15 15:00:56,775][52866] Updated weights for policy 1, policy_version 5340 (0.0007) -[2023-10-15 15:00:56,837][52833] Updated weights for policy 0, policy_version 5310 (0.0009) -[2023-10-15 15:00:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 10911744. Throughput: 0: 1738.4, 1: 1776.9. Samples: 2736400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:00:58,442][51532] Avg episode reward: [(0, '11.400'), (1, '12.060')] -[2023-10-15 15:01:00,512][52866] Updated weights for policy 1, policy_version 5350 (0.0007) -[2023-10-15 15:01:00,592][52833] Updated weights for policy 0, policy_version 5320 (0.0008) -[2023-10-15 15:01:00,877][52866] Updated weights for policy 1, policy_version 5360 (0.0008) -[2023-10-15 15:01:00,964][52833] Updated weights for policy 0, policy_version 5330 (0.0009) -[2023-10-15 15:01:01,250][52866] Updated weights for policy 1, policy_version 5370 (0.0009) -[2023-10-15 15:01:01,335][52833] Updated weights for policy 0, policy_version 5340 (0.0008) -[2023-10-15 15:01:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 10977280. Throughput: 0: 1757.7, 1: 1792.1. Samples: 2747660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:01:03,441][51532] Avg episode reward: [(0, '12.260'), (1, '11.870')] -[2023-10-15 15:01:03,442][52410] Saving new best policy, reward=12.260! -[2023-10-15 15:01:04,990][52866] Updated weights for policy 1, policy_version 5380 (0.0007) -[2023-10-15 15:01:05,204][52833] Updated weights for policy 0, policy_version 5350 (0.0008) -[2023-10-15 15:01:05,349][52866] Updated weights for policy 1, policy_version 5390 (0.0008) -[2023-10-15 15:01:05,570][52833] Updated weights for policy 0, policy_version 5360 (0.0008) -[2023-10-15 15:01:05,715][52866] Updated weights for policy 1, policy_version 5400 (0.0009) -[2023-10-15 15:01:05,941][52833] Updated weights for policy 0, policy_version 5370 (0.0009) -[2023-10-15 15:01:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 11042816. Throughput: 0: 1743.8, 1: 1774.8. Samples: 2768340. Policy #0 lag: (min: 4.0, avg: 14.2, max: 36.0) -[2023-10-15 15:01:08,442][51532] Avg episode reward: [(0, '11.880'), (1, '13.020')] -[2023-10-15 15:01:09,420][52866] Updated weights for policy 1, policy_version 5410 (0.0008) -[2023-10-15 15:01:09,795][52866] Updated weights for policy 1, policy_version 5420 (0.0009) -[2023-10-15 15:01:09,837][52833] Updated weights for policy 0, policy_version 5380 (0.0010) -[2023-10-15 15:01:10,151][52866] Updated weights for policy 1, policy_version 5430 (0.0007) -[2023-10-15 15:01:10,210][52833] Updated weights for policy 0, policy_version 5390 (0.0008) -[2023-10-15 15:01:10,509][52866] Updated weights for policy 1, policy_version 5440 (0.0008) -[2023-10-15 15:01:10,588][52833] Updated weights for policy 0, policy_version 5400 (0.0008) -[2023-10-15 15:01:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 11108352. Throughput: 0: 1746.0, 1: 1773.2. Samples: 2790396. Policy #0 lag: (min: 4.0, avg: 14.2, max: 36.0) -[2023-10-15 15:01:13,442][51532] Avg episode reward: [(0, '12.240'), (1, '13.080')] -[2023-10-15 15:01:14,320][52866] Updated weights for policy 1, policy_version 5450 (0.0007) -[2023-10-15 15:01:14,549][52833] Updated weights for policy 0, policy_version 5410 (0.0007) -[2023-10-15 15:01:14,682][52866] Updated weights for policy 1, policy_version 5460 (0.0007) -[2023-10-15 15:01:14,919][52833] Updated weights for policy 0, policy_version 5420 (0.0007) -[2023-10-15 15:01:15,044][52866] Updated weights for policy 1, policy_version 5470 (0.0007) -[2023-10-15 15:01:15,296][52833] Updated weights for policy 0, policy_version 5430 (0.0011) -[2023-10-15 15:01:15,674][52833] Updated weights for policy 0, policy_version 5440 (0.0011) -[2023-10-15 15:01:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 11173888. Throughput: 0: 1740.7, 1: 1776.1. Samples: 2799938. Policy #0 lag: (min: 2.0, avg: 4.2, max: 34.0) -[2023-10-15 15:01:18,442][51532] Avg episode reward: [(0, '12.120'), (1, '12.140')] -[2023-10-15 15:01:19,002][52866] Updated weights for policy 1, policy_version 5480 (0.0009) -[2023-10-15 15:01:19,369][52866] Updated weights for policy 1, policy_version 5490 (0.0009) -[2023-10-15 15:01:19,413][52833] Updated weights for policy 0, policy_version 5450 (0.0007) -[2023-10-15 15:01:19,735][52866] Updated weights for policy 1, policy_version 5500 (0.0009) -[2023-10-15 15:01:19,778][52833] Updated weights for policy 0, policy_version 5460 (0.0009) -[2023-10-15 15:01:20,151][52833] Updated weights for policy 0, policy_version 5470 (0.0011) -[2023-10-15 15:01:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 11239424. Throughput: 0: 1746.2, 1: 1773.3. Samples: 2821708. Policy #0 lag: (min: 2.0, avg: 4.2, max: 34.0) -[2023-10-15 15:01:23,442][51532] Avg episode reward: [(0, '10.930'), (1, '13.330')] -[2023-10-15 15:01:23,502][52866] Updated weights for policy 1, policy_version 5510 (0.0007) -[2023-10-15 15:01:23,869][52866] Updated weights for policy 1, policy_version 5520 (0.0007) -[2023-10-15 15:01:23,970][52833] Updated weights for policy 0, policy_version 5480 (0.0008) -[2023-10-15 15:01:24,237][52866] Updated weights for policy 1, policy_version 5530 (0.0008) -[2023-10-15 15:01:24,338][52833] Updated weights for policy 0, policy_version 5490 (0.0009) -[2023-10-15 15:01:24,715][52833] Updated weights for policy 0, policy_version 5500 (0.0007) -[2023-10-15 15:01:28,110][52866] Updated weights for policy 1, policy_version 5540 (0.0010) -[2023-10-15 15:01:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 11304960. Throughput: 0: 1778.5, 1: 1791.6. Samples: 2843582. Policy #0 lag: (min: 17.0, avg: 19.5, max: 43.0) -[2023-10-15 15:01:28,441][51532] Avg episode reward: [(0, '11.080'), (1, '12.490')] -[2023-10-15 15:01:28,475][52866] Updated weights for policy 1, policy_version 5550 (0.0010) -[2023-10-15 15:01:28,606][52833] Updated weights for policy 0, policy_version 5510 (0.0010) -[2023-10-15 15:01:28,831][52866] Updated weights for policy 1, policy_version 5560 (0.0007) -[2023-10-15 15:01:28,987][52833] Updated weights for policy 0, policy_version 5520 (0.0009) -[2023-10-15 15:01:29,353][52833] Updated weights for policy 0, policy_version 5530 (0.0009) -[2023-10-15 15:01:32,772][52866] Updated weights for policy 1, policy_version 5570 (0.0008) -[2023-10-15 15:01:33,143][52866] Updated weights for policy 1, policy_version 5580 (0.0008) -[2023-10-15 15:01:33,225][52833] Updated weights for policy 0, policy_version 5540 (0.0010) -[2023-10-15 15:01:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 11370496. Throughput: 0: 1749.6, 1: 1763.6. Samples: 2853058. Policy #0 lag: (min: 17.0, avg: 19.5, max: 43.0) -[2023-10-15 15:01:33,443][51532] Avg episode reward: [(0, '11.310'), (1, '13.080')] -[2023-10-15 15:01:33,512][52866] Updated weights for policy 1, policy_version 5590 (0.0009) -[2023-10-15 15:01:33,588][52833] Updated weights for policy 0, policy_version 5550 (0.0009) -[2023-10-15 15:01:33,880][52866] Updated weights for policy 1, policy_version 5600 (0.0008) -[2023-10-15 15:01:33,966][52833] Updated weights for policy 0, policy_version 5560 (0.0008) -[2023-10-15 15:01:37,735][52866] Updated weights for policy 1, policy_version 5610 (0.0011) -[2023-10-15 15:01:37,820][52833] Updated weights for policy 0, policy_version 5570 (0.0010) -[2023-10-15 15:01:38,093][52866] Updated weights for policy 1, policy_version 5620 (0.0007) -[2023-10-15 15:01:38,185][52833] Updated weights for policy 0, policy_version 5580 (0.0007) -[2023-10-15 15:01:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 14106.9). Total num frames: 11436032. Throughput: 0: 1764.0, 1: 1783.1. Samples: 2874966. Policy #0 lag: (min: 17.0, avg: 26.5, max: 49.0) -[2023-10-15 15:01:38,442][51532] Avg episode reward: [(0, '11.550'), (1, '13.580')] -[2023-10-15 15:01:38,460][52866] Updated weights for policy 1, policy_version 5630 (0.0007) -[2023-10-15 15:01:38,550][52833] Updated weights for policy 0, policy_version 5590 (0.0009) -[2023-10-15 15:01:38,913][52833] Updated weights for policy 0, policy_version 5600 (0.0007) -[2023-10-15 15:01:42,401][52866] Updated weights for policy 1, policy_version 5640 (0.0008) -[2023-10-15 15:01:42,765][52866] Updated weights for policy 1, policy_version 5650 (0.0007) -[2023-10-15 15:01:42,799][52833] Updated weights for policy 0, policy_version 5610 (0.0007) -[2023-10-15 15:01:43,136][52866] Updated weights for policy 1, policy_version 5660 (0.0007) -[2023-10-15 15:01:43,159][52833] Updated weights for policy 0, policy_version 5620 (0.0007) -[2023-10-15 15:01:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 13653.4, 300 sec: 14218.0). Total num frames: 11534336. Throughput: 0: 1765.2, 1: 1769.9. Samples: 2895480. Policy #0 lag: (min: 17.0, avg: 26.5, max: 49.0) -[2023-10-15 15:01:43,441][51532] Avg episode reward: [(0, '12.690'), (1, '13.010')] -[2023-10-15 15:01:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000005664_5799936.pth... -[2023-10-15 15:01:43,481][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000004000_4096000.pth -[2023-10-15 15:01:43,531][52833] Updated weights for policy 0, policy_version 5630 (0.0007) -[2023-10-15 15:01:43,601][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000005632_5767168.pth... -[2023-10-15 15:01:43,640][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000003968_4063232.pth -[2023-10-15 15:01:43,645][52410] Saving new best policy, reward=12.690! -[2023-10-15 15:01:47,054][52866] Updated weights for policy 1, policy_version 5670 (0.0009) -[2023-10-15 15:01:47,359][52833] Updated weights for policy 0, policy_version 5640 (0.0010) -[2023-10-15 15:01:47,421][52866] Updated weights for policy 1, policy_version 5680 (0.0009) -[2023-10-15 15:01:47,721][52833] Updated weights for policy 0, policy_version 5650 (0.0008) -[2023-10-15 15:01:47,784][52866] Updated weights for policy 1, policy_version 5690 (0.0007) -[2023-10-15 15:01:48,095][52833] Updated weights for policy 0, policy_version 5660 (0.0007) -[2023-10-15 15:01:48,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 11632640. Throughput: 0: 1756.8, 1: 1770.9. Samples: 2906408. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:01:48,442][51532] Avg episode reward: [(0, '12.400'), (1, '13.870')] -[2023-10-15 15:01:51,706][52866] Updated weights for policy 1, policy_version 5700 (0.0008) -[2023-10-15 15:01:51,808][52833] Updated weights for policy 0, policy_version 5670 (0.0007) -[2023-10-15 15:01:52,070][52866] Updated weights for policy 1, policy_version 5710 (0.0007) -[2023-10-15 15:01:52,173][52833] Updated weights for policy 0, policy_version 5680 (0.0009) -[2023-10-15 15:01:52,446][52866] Updated weights for policy 1, policy_version 5720 (0.0008) -[2023-10-15 15:01:52,552][52833] Updated weights for policy 0, policy_version 5690 (0.0008) -[2023-10-15 15:01:53,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 11698176. Throughput: 0: 1774.7, 1: 1769.2. Samples: 2927818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:01:53,442][51532] Avg episode reward: [(0, '12.440'), (1, '13.300')] -[2023-10-15 15:01:56,032][52866] Updated weights for policy 1, policy_version 5730 (0.0008) -[2023-10-15 15:01:56,405][52866] Updated weights for policy 1, policy_version 5740 (0.0009) -[2023-10-15 15:01:56,420][52833] Updated weights for policy 0, policy_version 5700 (0.0007) -[2023-10-15 15:01:56,767][52866] Updated weights for policy 1, policy_version 5750 (0.0008) -[2023-10-15 15:01:56,798][52833] Updated weights for policy 0, policy_version 5710 (0.0008) -[2023-10-15 15:01:57,137][52866] Updated weights for policy 1, policy_version 5760 (0.0008) -[2023-10-15 15:01:57,173][52833] Updated weights for policy 0, policy_version 5720 (0.0009) -[2023-10-15 15:01:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 11763712. Throughput: 0: 1751.3, 1: 1747.7. Samples: 2947850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:01:58,442][51532] Avg episode reward: [(0, '10.450'), (1, '13.060')] -[2023-10-15 15:02:01,024][52833] Updated weights for policy 0, policy_version 5730 (0.0010) -[2023-10-15 15:02:01,064][52866] Updated weights for policy 1, policy_version 5770 (0.0009) -[2023-10-15 15:02:01,398][52833] Updated weights for policy 0, policy_version 5740 (0.0008) -[2023-10-15 15:02:01,432][52866] Updated weights for policy 1, policy_version 5780 (0.0010) -[2023-10-15 15:02:01,775][52833] Updated weights for policy 0, policy_version 5750 (0.0009) -[2023-10-15 15:02:01,801][52866] Updated weights for policy 1, policy_version 5790 (0.0007) -[2023-10-15 15:02:02,138][52833] Updated weights for policy 0, policy_version 5760 (0.0008) -[2023-10-15 15:02:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 11829248. Throughput: 0: 1784.6, 1: 1768.9. Samples: 2959846. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 15:02:03,442][51532] Avg episode reward: [(0, '11.400'), (1, '12.170')] -[2023-10-15 15:02:05,641][52866] Updated weights for policy 1, policy_version 5800 (0.0010) -[2023-10-15 15:02:06,003][52866] Updated weights for policy 1, policy_version 5810 (0.0007) -[2023-10-15 15:02:06,101][52833] Updated weights for policy 0, policy_version 5770 (0.0008) -[2023-10-15 15:02:06,374][52866] Updated weights for policy 1, policy_version 5820 (0.0007) -[2023-10-15 15:02:06,475][52833] Updated weights for policy 0, policy_version 5780 (0.0008) -[2023-10-15 15:02:06,836][52833] Updated weights for policy 0, policy_version 5790 (0.0011) -[2023-10-15 15:02:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 11894784. Throughput: 0: 1752.4, 1: 1747.7. Samples: 2979210. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 15:02:08,442][51532] Avg episode reward: [(0, '10.450'), (1, '12.280')] -[2023-10-15 15:02:10,231][52866] Updated weights for policy 1, policy_version 5830 (0.0008) -[2023-10-15 15:02:10,448][52833] Updated weights for policy 0, policy_version 5800 (0.0009) -[2023-10-15 15:02:10,599][52866] Updated weights for policy 1, policy_version 5840 (0.0007) -[2023-10-15 15:02:10,827][52833] Updated weights for policy 0, policy_version 5810 (0.0008) -[2023-10-15 15:02:10,965][52866] Updated weights for policy 1, policy_version 5850 (0.0009) -[2023-10-15 15:02:11,197][52833] Updated weights for policy 0, policy_version 5820 (0.0008) -[2023-10-15 15:02:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 11960320. Throughput: 0: 1752.0, 1: 1746.9. Samples: 3001034. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:02:13,442][51532] Avg episode reward: [(0, '11.380'), (1, '12.430')] -[2023-10-15 15:02:14,852][52866] Updated weights for policy 1, policy_version 5860 (0.0007) -[2023-10-15 15:02:15,093][52833] Updated weights for policy 0, policy_version 5830 (0.0010) -[2023-10-15 15:02:15,212][52866] Updated weights for policy 1, policy_version 5870 (0.0009) -[2023-10-15 15:02:15,473][52833] Updated weights for policy 0, policy_version 5840 (0.0010) -[2023-10-15 15:02:15,580][52866] Updated weights for policy 1, policy_version 5880 (0.0008) -[2023-10-15 15:02:15,846][52833] Updated weights for policy 0, policy_version 5850 (0.0009) -[2023-10-15 15:02:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 12025856. Throughput: 0: 1756.6, 1: 1745.4. Samples: 3010650. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:02:18,441][51532] Avg episode reward: [(0, '10.590'), (1, '13.110')] -[2023-10-15 15:02:19,376][52866] Updated weights for policy 1, policy_version 5890 (0.0009) -[2023-10-15 15:02:19,576][52833] Updated weights for policy 0, policy_version 5860 (0.0010) -[2023-10-15 15:02:19,749][52866] Updated weights for policy 1, policy_version 5900 (0.0008) -[2023-10-15 15:02:19,952][52833] Updated weights for policy 0, policy_version 5870 (0.0008) -[2023-10-15 15:02:20,106][52866] Updated weights for policy 1, policy_version 5910 (0.0008) -[2023-10-15 15:02:20,308][52833] Updated weights for policy 0, policy_version 5880 (0.0008) -[2023-10-15 15:02:20,474][52866] Updated weights for policy 1, policy_version 5920 (0.0009) -[2023-10-15 15:02:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 12091392. Throughput: 0: 1752.2, 1: 1745.8. Samples: 3032376. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 15:02:23,441][51532] Avg episode reward: [(0, '12.010'), (1, '13.440')] -[2023-10-15 15:02:24,031][52833] Updated weights for policy 0, policy_version 5890 (0.0008) -[2023-10-15 15:02:24,298][52866] Updated weights for policy 1, policy_version 5930 (0.0007) -[2023-10-15 15:02:24,395][52833] Updated weights for policy 0, policy_version 5900 (0.0008) -[2023-10-15 15:02:24,666][52866] Updated weights for policy 1, policy_version 5940 (0.0007) -[2023-10-15 15:02:24,757][52833] Updated weights for policy 0, policy_version 5910 (0.0007) -[2023-10-15 15:02:25,039][52866] Updated weights for policy 1, policy_version 5950 (0.0007) -[2023-10-15 15:02:25,124][52833] Updated weights for policy 0, policy_version 5920 (0.0009) -[2023-10-15 15:02:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 12156928. Throughput: 0: 1761.9, 1: 1779.3. Samples: 3054830. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 15:02:28,441][51532] Avg episode reward: [(0, '12.430'), (1, '12.310')] -[2023-10-15 15:02:28,681][52866] Updated weights for policy 1, policy_version 5960 (0.0010) -[2023-10-15 15:02:29,050][52833] Updated weights for policy 0, policy_version 5930 (0.0007) -[2023-10-15 15:02:29,054][52866] Updated weights for policy 1, policy_version 5970 (0.0008) -[2023-10-15 15:02:29,413][52833] Updated weights for policy 0, policy_version 5940 (0.0007) -[2023-10-15 15:02:29,425][52866] Updated weights for policy 1, policy_version 5980 (0.0010) -[2023-10-15 15:02:29,788][52833] Updated weights for policy 0, policy_version 5950 (0.0008) -[2023-10-15 15:02:33,282][52866] Updated weights for policy 1, policy_version 5990 (0.0008) -[2023-10-15 15:02:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 12222464. Throughput: 0: 1753.8, 1: 1761.6. Samples: 3064598. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) -[2023-10-15 15:02:33,441][51532] Avg episode reward: [(0, '13.020'), (1, '11.420')] -[2023-10-15 15:02:33,512][52833] Updated weights for policy 0, policy_version 5960 (0.0008) -[2023-10-15 15:02:33,651][52866] Updated weights for policy 1, policy_version 6000 (0.0007) -[2023-10-15 15:02:33,876][52833] Updated weights for policy 0, policy_version 5970 (0.0008) -[2023-10-15 15:02:34,016][52866] Updated weights for policy 1, policy_version 6010 (0.0009) -[2023-10-15 15:02:34,242][52833] Updated weights for policy 0, policy_version 5980 (0.0008) -[2023-10-15 15:02:34,389][52410] Saving new best policy, reward=13.020! -[2023-10-15 15:02:37,781][52866] Updated weights for policy 1, policy_version 6020 (0.0007) -[2023-10-15 15:02:38,143][52866] Updated weights for policy 1, policy_version 6030 (0.0007) -[2023-10-15 15:02:38,162][52833] Updated weights for policy 0, policy_version 5990 (0.0009) -[2023-10-15 15:02:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 12288000. Throughput: 0: 1751.5, 1: 1778.0. Samples: 3086644. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) -[2023-10-15 15:02:38,441][51532] Avg episode reward: [(0, '12.740'), (1, '11.060')] -[2023-10-15 15:02:38,516][52866] Updated weights for policy 1, policy_version 6040 (0.0009) -[2023-10-15 15:02:38,530][52833] Updated weights for policy 0, policy_version 6000 (0.0008) -[2023-10-15 15:02:38,905][52833] Updated weights for policy 0, policy_version 6010 (0.0007) -[2023-10-15 15:02:42,348][52866] Updated weights for policy 1, policy_version 6050 (0.0008) -[2023-10-15 15:02:42,698][52833] Updated weights for policy 0, policy_version 6020 (0.0009) -[2023-10-15 15:02:42,717][52866] Updated weights for policy 1, policy_version 6060 (0.0008) -[2023-10-15 15:02:43,078][52833] Updated weights for policy 0, policy_version 6030 (0.0007) -[2023-10-15 15:02:43,086][52866] Updated weights for policy 1, policy_version 6070 (0.0007) -[2023-10-15 15:02:43,440][52833] Updated weights for policy 0, policy_version 6040 (0.0009) -[2023-10-15 15:02:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 14106.9). Total num frames: 12353536. Throughput: 0: 1772.8, 1: 1775.3. Samples: 3107516. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) -[2023-10-15 15:02:43,441][51532] Avg episode reward: [(0, '12.320'), (1, '11.540')] -[2023-10-15 15:02:43,462][52866] Updated weights for policy 1, policy_version 6080 (0.0007) -[2023-10-15 15:02:47,168][52866] Updated weights for policy 1, policy_version 6090 (0.0007) -[2023-10-15 15:02:47,354][52833] Updated weights for policy 0, policy_version 6050 (0.0010) -[2023-10-15 15:02:47,533][52866] Updated weights for policy 1, policy_version 6100 (0.0008) -[2023-10-15 15:02:47,718][52833] Updated weights for policy 0, policy_version 6060 (0.0009) -[2023-10-15 15:02:47,899][52866] Updated weights for policy 1, policy_version 6110 (0.0008) -[2023-10-15 15:02:48,089][52833] Updated weights for policy 0, policy_version 6070 (0.0009) -[2023-10-15 15:02:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 14106.9). Total num frames: 12451840. Throughput: 0: 1749.3, 1: 1772.5. Samples: 3118326. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) -[2023-10-15 15:02:48,442][51532] Avg episode reward: [(0, '12.020'), (1, '12.450')] -[2023-10-15 15:02:48,459][52833] Updated weights for policy 0, policy_version 6080 (0.0008) -[2023-10-15 15:02:51,649][52866] Updated weights for policy 1, policy_version 6120 (0.0009) -[2023-10-15 15:02:52,015][52866] Updated weights for policy 1, policy_version 6130 (0.0008) -[2023-10-15 15:02:52,370][52866] Updated weights for policy 1, policy_version 6140 (0.0009) -[2023-10-15 15:02:52,462][52833] Updated weights for policy 0, policy_version 6090 (0.0007) -[2023-10-15 15:02:52,846][52833] Updated weights for policy 0, policy_version 6100 (0.0008) -[2023-10-15 15:02:53,216][52833] Updated weights for policy 0, policy_version 6110 (0.0007) -[2023-10-15 15:02:53,441][51532] Fps is (10 sec: 19660.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 12550144. Throughput: 0: 1781.0, 1: 1784.0. Samples: 3139634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:02:53,442][51532] Avg episode reward: [(0, '11.960'), (1, '12.660')] -[2023-10-15 15:02:56,045][52866] Updated weights for policy 1, policy_version 6150 (0.0007) -[2023-10-15 15:02:56,410][52866] Updated weights for policy 1, policy_version 6160 (0.0009) -[2023-10-15 15:02:56,772][52866] Updated weights for policy 1, policy_version 6170 (0.0008) -[2023-10-15 15:02:56,799][52833] Updated weights for policy 0, policy_version 6120 (0.0008) -[2023-10-15 15:02:57,174][52833] Updated weights for policy 0, policy_version 6130 (0.0009) -[2023-10-15 15:02:57,541][52833] Updated weights for policy 0, policy_version 6140 (0.0009) -[2023-10-15 15:02:58,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 12615680. Throughput: 0: 1755.8, 1: 1781.8. Samples: 3160226. Policy #0 lag: (min: 10.0, avg: 18.3, max: 42.0) -[2023-10-15 15:02:58,442][51532] Avg episode reward: [(0, '12.370'), (1, '12.850')] -[2023-10-15 15:03:00,587][52866] Updated weights for policy 1, policy_version 6180 (0.0009) -[2023-10-15 15:03:00,959][52866] Updated weights for policy 1, policy_version 6190 (0.0007) -[2023-10-15 15:03:01,322][52866] Updated weights for policy 1, policy_version 6200 (0.0009) -[2023-10-15 15:03:01,348][52833] Updated weights for policy 0, policy_version 6150 (0.0010) -[2023-10-15 15:03:01,727][52833] Updated weights for policy 0, policy_version 6160 (0.0009) -[2023-10-15 15:03:02,103][52833] Updated weights for policy 0, policy_version 6170 (0.0009) -[2023-10-15 15:03:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 12681216. Throughput: 0: 1788.6, 1: 1796.8. Samples: 3171994. Policy #0 lag: (min: 10.0, avg: 18.3, max: 42.0) -[2023-10-15 15:03:03,441][51532] Avg episode reward: [(0, '12.550'), (1, '13.100')] -[2023-10-15 15:03:05,139][52866] Updated weights for policy 1, policy_version 6210 (0.0007) -[2023-10-15 15:03:05,505][52866] Updated weights for policy 1, policy_version 6220 (0.0009) -[2023-10-15 15:03:05,720][52833] Updated weights for policy 0, policy_version 6180 (0.0009) -[2023-10-15 15:03:05,878][52866] Updated weights for policy 1, policy_version 6230 (0.0009) -[2023-10-15 15:03:06,084][52833] Updated weights for policy 0, policy_version 6190 (0.0008) -[2023-10-15 15:03:06,238][52866] Updated weights for policy 1, policy_version 6240 (0.0009) -[2023-10-15 15:03:06,451][52833] Updated weights for policy 0, policy_version 6200 (0.0010) -[2023-10-15 15:03:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 12746752. Throughput: 0: 1763.3, 1: 1782.3. Samples: 3191928. Policy #0 lag: (min: 30.0, avg: 35.6, max: 62.0) -[2023-10-15 15:03:08,442][51532] Avg episode reward: [(0, '12.350'), (1, '13.950')] -[2023-10-15 15:03:10,034][52866] Updated weights for policy 1, policy_version 6250 (0.0010) -[2023-10-15 15:03:10,350][52833] Updated weights for policy 0, policy_version 6210 (0.0010) -[2023-10-15 15:03:10,391][52866] Updated weights for policy 1, policy_version 6260 (0.0009) -[2023-10-15 15:03:10,725][52833] Updated weights for policy 0, policy_version 6220 (0.0009) -[2023-10-15 15:03:10,766][52866] Updated weights for policy 1, policy_version 6270 (0.0009) -[2023-10-15 15:03:11,088][52833] Updated weights for policy 0, policy_version 6230 (0.0010) -[2023-10-15 15:03:11,456][52833] Updated weights for policy 0, policy_version 6240 (0.0008) -[2023-10-15 15:03:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 12812288. Throughput: 0: 1765.7, 1: 1776.8. Samples: 3214244. Policy #0 lag: (min: 30.0, avg: 35.6, max: 62.0) -[2023-10-15 15:03:13,442][51532] Avg episode reward: [(0, '11.170'), (1, '13.300')] -[2023-10-15 15:03:14,577][52866] Updated weights for policy 1, policy_version 6280 (0.0007) -[2023-10-15 15:03:14,934][52866] Updated weights for policy 1, policy_version 6290 (0.0008) -[2023-10-15 15:03:15,179][52833] Updated weights for policy 0, policy_version 6250 (0.0008) -[2023-10-15 15:03:15,304][52866] Updated weights for policy 1, policy_version 6300 (0.0008) -[2023-10-15 15:03:15,544][52833] Updated weights for policy 0, policy_version 6260 (0.0008) -[2023-10-15 15:03:15,911][52833] Updated weights for policy 0, policy_version 6270 (0.0009) -[2023-10-15 15:03:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 12877824. Throughput: 0: 1774.4, 1: 1776.2. Samples: 3224374. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-15 15:03:18,442][51532] Avg episode reward: [(0, '11.250'), (1, '13.950')] -[2023-10-15 15:03:19,186][52866] Updated weights for policy 1, policy_version 6310 (0.0007) -[2023-10-15 15:03:19,550][52866] Updated weights for policy 1, policy_version 6320 (0.0007) -[2023-10-15 15:03:19,727][52833] Updated weights for policy 0, policy_version 6280 (0.0009) -[2023-10-15 15:03:19,919][52866] Updated weights for policy 1, policy_version 6330 (0.0007) -[2023-10-15 15:03:20,102][52833] Updated weights for policy 0, policy_version 6290 (0.0007) -[2023-10-15 15:03:20,471][52833] Updated weights for policy 0, policy_version 6300 (0.0009) -[2023-10-15 15:03:23,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 12943360. Throughput: 0: 1768.7, 1: 1773.0. Samples: 3246020. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-15 15:03:23,441][51532] Avg episode reward: [(0, '11.150'), (1, '13.840')] -[2023-10-15 15:03:23,869][52866] Updated weights for policy 1, policy_version 6340 (0.0009) -[2023-10-15 15:03:24,231][52866] Updated weights for policy 1, policy_version 6350 (0.0009) -[2023-10-15 15:03:24,277][52833] Updated weights for policy 0, policy_version 6310 (0.0007) -[2023-10-15 15:03:24,595][52866] Updated weights for policy 1, policy_version 6360 (0.0008) -[2023-10-15 15:03:24,643][52833] Updated weights for policy 0, policy_version 6320 (0.0008) -[2023-10-15 15:03:25,015][52833] Updated weights for policy 0, policy_version 6330 (0.0009) -[2023-10-15 15:03:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 13008896. Throughput: 0: 1781.7, 1: 1786.8. Samples: 3268100. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 15:03:28,441][51532] Avg episode reward: [(0, '11.630'), (1, '12.850')] -[2023-10-15 15:03:28,469][52866] Updated weights for policy 1, policy_version 6370 (0.0008) -[2023-10-15 15:03:28,827][52833] Updated weights for policy 0, policy_version 6340 (0.0007) -[2023-10-15 15:03:28,843][52866] Updated weights for policy 1, policy_version 6380 (0.0009) -[2023-10-15 15:03:29,194][52833] Updated weights for policy 0, policy_version 6350 (0.0007) -[2023-10-15 15:03:29,209][52866] Updated weights for policy 1, policy_version 6390 (0.0008) -[2023-10-15 15:03:29,560][52833] Updated weights for policy 0, policy_version 6360 (0.0007) -[2023-10-15 15:03:29,581][52866] Updated weights for policy 1, policy_version 6400 (0.0009) -[2023-10-15 15:03:33,179][52866] Updated weights for policy 1, policy_version 6410 (0.0009) -[2023-10-15 15:03:33,354][52833] Updated weights for policy 0, policy_version 6370 (0.0009) -[2023-10-15 15:03:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 13074432. Throughput: 0: 1775.7, 1: 1770.3. Samples: 3277894. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 15:03:33,442][51532] Avg episode reward: [(0, '12.030'), (1, '14.110')] -[2023-10-15 15:03:33,556][52866] Updated weights for policy 1, policy_version 6420 (0.0009) -[2023-10-15 15:03:33,723][52833] Updated weights for policy 0, policy_version 6380 (0.0008) -[2023-10-15 15:03:33,928][52866] Updated weights for policy 1, policy_version 6430 (0.0008) -[2023-10-15 15:03:34,092][52833] Updated weights for policy 0, policy_version 6390 (0.0007) -[2023-10-15 15:03:34,463][52833] Updated weights for policy 0, policy_version 6400 (0.0007) -[2023-10-15 15:03:37,883][52866] Updated weights for policy 1, policy_version 6440 (0.0008) -[2023-10-15 15:03:38,249][52866] Updated weights for policy 1, policy_version 6450 (0.0008) -[2023-10-15 15:03:38,422][52833] Updated weights for policy 0, policy_version 6410 (0.0008) -[2023-10-15 15:03:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 13139968. Throughput: 0: 1777.2, 1: 1783.7. Samples: 3299874. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-15 15:03:38,441][51532] Avg episode reward: [(0, '13.320'), (1, '12.800')] -[2023-10-15 15:03:38,616][52866] Updated weights for policy 1, policy_version 6460 (0.0009) -[2023-10-15 15:03:38,797][52833] Updated weights for policy 0, policy_version 6420 (0.0008) -[2023-10-15 15:03:39,181][52833] Updated weights for policy 0, policy_version 6430 (0.0008) -[2023-10-15 15:03:39,245][52410] Saving new best policy, reward=13.320! -[2023-10-15 15:03:42,329][52866] Updated weights for policy 1, policy_version 6470 (0.0008) -[2023-10-15 15:03:42,692][52866] Updated weights for policy 1, policy_version 6480 (0.0009) -[2023-10-15 15:03:42,898][52833] Updated weights for policy 0, policy_version 6440 (0.0008) -[2023-10-15 15:03:43,062][52866] Updated weights for policy 1, policy_version 6490 (0.0009) -[2023-10-15 15:03:43,267][52833] Updated weights for policy 0, policy_version 6450 (0.0008) -[2023-10-15 15:03:43,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14106.9). Total num frames: 13238272. Throughput: 0: 1795.2, 1: 1769.5. Samples: 3320634. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-15 15:03:43,441][51532] Avg episode reward: [(0, '13.370'), (1, '13.180')] -[2023-10-15 15:03:43,449][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000006496_6651904.pth... -[2023-10-15 15:03:43,478][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000004832_4947968.pth -[2023-10-15 15:03:43,643][52833] Updated weights for policy 0, policy_version 6460 (0.0009) -[2023-10-15 15:03:43,792][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000006464_6619136.pth... -[2023-10-15 15:03:43,820][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000004800_4915200.pth -[2023-10-15 15:03:43,824][52410] Saving new best policy, reward=13.370! -[2023-10-15 15:03:46,871][52866] Updated weights for policy 1, policy_version 6500 (0.0009) -[2023-10-15 15:03:47,241][52866] Updated weights for policy 1, policy_version 6510 (0.0009) -[2023-10-15 15:03:47,454][52833] Updated weights for policy 0, policy_version 6470 (0.0007) -[2023-10-15 15:03:47,612][52866] Updated weights for policy 1, policy_version 6520 (0.0008) -[2023-10-15 15:03:47,832][52833] Updated weights for policy 0, policy_version 6480 (0.0009) -[2023-10-15 15:03:48,201][52833] Updated weights for policy 0, policy_version 6490 (0.0007) -[2023-10-15 15:03:48,441][51532] Fps is (10 sec: 19660.4, 60 sec: 14745.6, 300 sec: 14218.0). Total num frames: 13336576. Throughput: 0: 1772.6, 1: 1778.4. Samples: 3331792. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-15 15:03:48,442][51532] Avg episode reward: [(0, '12.810'), (1, '12.320')] -[2023-10-15 15:03:51,374][52866] Updated weights for policy 1, policy_version 6530 (0.0008) -[2023-10-15 15:03:51,746][52866] Updated weights for policy 1, policy_version 6540 (0.0008) -[2023-10-15 15:03:51,898][52833] Updated weights for policy 0, policy_version 6500 (0.0008) -[2023-10-15 15:03:52,112][52866] Updated weights for policy 1, policy_version 6550 (0.0007) -[2023-10-15 15:03:52,259][52833] Updated weights for policy 0, policy_version 6510 (0.0007) -[2023-10-15 15:03:52,473][52866] Updated weights for policy 1, policy_version 6560 (0.0008) -[2023-10-15 15:03:52,630][52833] Updated weights for policy 0, policy_version 6520 (0.0007) -[2023-10-15 15:03:53,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 13402112. Throughput: 0: 1801.2, 1: 1779.1. Samples: 3353040. Policy #0 lag: (min: 7.0, avg: 29.3, max: 32.0) -[2023-10-15 15:03:53,442][51532] Avg episode reward: [(0, '12.810'), (1, '11.900')] -[2023-10-15 15:03:56,395][52866] Updated weights for policy 1, policy_version 6570 (0.0009) -[2023-10-15 15:03:56,482][52833] Updated weights for policy 0, policy_version 6530 (0.0007) -[2023-10-15 15:03:56,766][52866] Updated weights for policy 1, policy_version 6580 (0.0008) -[2023-10-15 15:03:56,843][52833] Updated weights for policy 0, policy_version 6540 (0.0008) -[2023-10-15 15:03:57,129][52866] Updated weights for policy 1, policy_version 6590 (0.0008) -[2023-10-15 15:03:57,210][52833] Updated weights for policy 0, policy_version 6550 (0.0009) -[2023-10-15 15:03:57,580][52833] Updated weights for policy 0, policy_version 6560 (0.0009) -[2023-10-15 15:03:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 13467648. Throughput: 0: 1770.5, 1: 1763.0. Samples: 3373252. Policy #0 lag: (min: 7.0, avg: 29.3, max: 32.0) -[2023-10-15 15:03:58,442][51532] Avg episode reward: [(0, '12.610'), (1, '12.640')] -[2023-10-15 15:04:00,968][52866] Updated weights for policy 1, policy_version 6600 (0.0008) -[2023-10-15 15:04:01,328][52866] Updated weights for policy 1, policy_version 6610 (0.0009) -[2023-10-15 15:04:01,447][52833] Updated weights for policy 0, policy_version 6570 (0.0008) -[2023-10-15 15:04:01,692][52866] Updated weights for policy 1, policy_version 6620 (0.0010) -[2023-10-15 15:04:01,819][52833] Updated weights for policy 0, policy_version 6580 (0.0008) -[2023-10-15 15:04:02,190][52833] Updated weights for policy 0, policy_version 6590 (0.0009) -[2023-10-15 15:04:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 13533184. Throughput: 0: 1794.0, 1: 1784.9. Samples: 3385428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:04:03,442][51532] Avg episode reward: [(0, '11.250'), (1, '13.700')] -[2023-10-15 15:04:05,434][52866] Updated weights for policy 1, policy_version 6630 (0.0008) -[2023-10-15 15:04:05,812][52866] Updated weights for policy 1, policy_version 6640 (0.0009) -[2023-10-15 15:04:05,838][52833] Updated weights for policy 0, policy_version 6600 (0.0009) -[2023-10-15 15:04:06,186][52866] Updated weights for policy 1, policy_version 6650 (0.0008) -[2023-10-15 15:04:06,210][52833] Updated weights for policy 0, policy_version 6610 (0.0007) -[2023-10-15 15:04:06,579][52833] Updated weights for policy 0, policy_version 6620 (0.0009) -[2023-10-15 15:04:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 13598720. Throughput: 0: 1769.9, 1: 1763.7. Samples: 3405030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:04:08,441][51532] Avg episode reward: [(0, '11.090'), (1, '12.150')] -[2023-10-15 15:04:09,959][52866] Updated weights for policy 1, policy_version 6660 (0.0009) -[2023-10-15 15:04:10,322][52866] Updated weights for policy 1, policy_version 6670 (0.0008) -[2023-10-15 15:04:10,345][52833] Updated weights for policy 0, policy_version 6630 (0.0007) -[2023-10-15 15:04:10,688][52866] Updated weights for policy 1, policy_version 6680 (0.0007) -[2023-10-15 15:04:10,720][52833] Updated weights for policy 0, policy_version 6640 (0.0008) -[2023-10-15 15:04:11,083][52833] Updated weights for policy 0, policy_version 6650 (0.0008) -[2023-10-15 15:04:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 13664256. Throughput: 0: 1767.5, 1: 1771.2. Samples: 3427338. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 15:04:13,442][51532] Avg episode reward: [(0, '11.300'), (1, '13.300')] -[2023-10-15 15:04:14,437][52866] Updated weights for policy 1, policy_version 6690 (0.0009) -[2023-10-15 15:04:14,803][52866] Updated weights for policy 1, policy_version 6700 (0.0009) -[2023-10-15 15:04:14,910][52833] Updated weights for policy 0, policy_version 6660 (0.0007) -[2023-10-15 15:04:15,166][52866] Updated weights for policy 1, policy_version 6710 (0.0008) -[2023-10-15 15:04:15,273][52833] Updated weights for policy 0, policy_version 6670 (0.0008) -[2023-10-15 15:04:15,531][52866] Updated weights for policy 1, policy_version 6720 (0.0008) -[2023-10-15 15:04:15,650][52833] Updated weights for policy 0, policy_version 6680 (0.0009) -[2023-10-15 15:04:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 13729792. Throughput: 0: 1770.0, 1: 1768.4. Samples: 3437120. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 15:04:18,442][51532] Avg episode reward: [(0, '13.000'), (1, '12.590')] -[2023-10-15 15:04:19,374][52866] Updated weights for policy 1, policy_version 6730 (0.0008) -[2023-10-15 15:04:19,446][52833] Updated weights for policy 0, policy_version 6690 (0.0007) -[2023-10-15 15:04:19,731][52866] Updated weights for policy 1, policy_version 6740 (0.0008) -[2023-10-15 15:04:19,816][52833] Updated weights for policy 0, policy_version 6700 (0.0007) -[2023-10-15 15:04:20,106][52866] Updated weights for policy 1, policy_version 6750 (0.0007) -[2023-10-15 15:04:20,186][52833] Updated weights for policy 0, policy_version 6710 (0.0007) -[2023-10-15 15:04:20,554][52833] Updated weights for policy 0, policy_version 6720 (0.0011) -[2023-10-15 15:04:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 13795328. Throughput: 0: 1767.5, 1: 1766.4. Samples: 3458902. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:04:23,442][51532] Avg episode reward: [(0, '12.640'), (1, '12.480')] -[2023-10-15 15:04:24,074][52866] Updated weights for policy 1, policy_version 6760 (0.0009) -[2023-10-15 15:04:24,438][52866] Updated weights for policy 1, policy_version 6770 (0.0008) -[2023-10-15 15:04:24,497][52833] Updated weights for policy 0, policy_version 6730 (0.0007) -[2023-10-15 15:04:24,808][52866] Updated weights for policy 1, policy_version 6780 (0.0007) -[2023-10-15 15:04:24,873][52833] Updated weights for policy 0, policy_version 6740 (0.0008) -[2023-10-15 15:04:25,247][52833] Updated weights for policy 0, policy_version 6750 (0.0010) -[2023-10-15 15:04:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 13860864. Throughput: 0: 1772.2, 1: 1784.5. Samples: 3480686. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:04:28,441][51532] Avg episode reward: [(0, '14.030'), (1, '12.160')] -[2023-10-15 15:04:28,451][52410] Saving new best policy, reward=14.030! -[2023-10-15 15:04:28,592][52866] Updated weights for policy 1, policy_version 6790 (0.0009) -[2023-10-15 15:04:28,939][52833] Updated weights for policy 0, policy_version 6760 (0.0008) -[2023-10-15 15:04:28,961][52866] Updated weights for policy 1, policy_version 6800 (0.0008) -[2023-10-15 15:04:29,302][52833] Updated weights for policy 0, policy_version 6770 (0.0008) -[2023-10-15 15:04:29,337][52866] Updated weights for policy 1, policy_version 6810 (0.0009) -[2023-10-15 15:04:29,669][52833] Updated weights for policy 0, policy_version 6780 (0.0008) -[2023-10-15 15:04:33,100][52866] Updated weights for policy 1, policy_version 6820 (0.0008) -[2023-10-15 15:04:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 13926400. Throughput: 0: 1760.1, 1: 1761.0. Samples: 3490242. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:04:33,441][51532] Avg episode reward: [(0, '13.410'), (1, '12.910')] -[2023-10-15 15:04:33,471][52866] Updated weights for policy 1, policy_version 6830 (0.0008) -[2023-10-15 15:04:33,547][52833] Updated weights for policy 0, policy_version 6790 (0.0007) -[2023-10-15 15:04:33,833][52866] Updated weights for policy 1, policy_version 6840 (0.0008) -[2023-10-15 15:04:33,913][52833] Updated weights for policy 0, policy_version 6800 (0.0007) -[2023-10-15 15:04:34,276][52833] Updated weights for policy 0, policy_version 6810 (0.0010) -[2023-10-15 15:04:37,676][52866] Updated weights for policy 1, policy_version 6850 (0.0008) -[2023-10-15 15:04:38,042][52833] Updated weights for policy 0, policy_version 6820 (0.0007) -[2023-10-15 15:04:38,044][52866] Updated weights for policy 1, policy_version 6860 (0.0007) -[2023-10-15 15:04:38,401][52866] Updated weights for policy 1, policy_version 6870 (0.0008) -[2023-10-15 15:04:38,405][52833] Updated weights for policy 0, policy_version 6830 (0.0009) -[2023-10-15 15:04:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 13991936. Throughput: 0: 1765.3, 1: 1775.4. Samples: 3512370. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:04:38,441][51532] Avg episode reward: [(0, '12.400'), (1, '13.540')] -[2023-10-15 15:04:38,769][52866] Updated weights for policy 1, policy_version 6880 (0.0007) -[2023-10-15 15:04:38,771][52833] Updated weights for policy 0, policy_version 6840 (0.0009) -[2023-10-15 15:04:42,609][52866] Updated weights for policy 1, policy_version 6890 (0.0007) -[2023-10-15 15:04:42,699][52833] Updated weights for policy 0, policy_version 6850 (0.0008) -[2023-10-15 15:04:42,965][52866] Updated weights for policy 1, policy_version 6900 (0.0007) -[2023-10-15 15:04:43,070][52833] Updated weights for policy 0, policy_version 6860 (0.0008) -[2023-10-15 15:04:43,335][52866] Updated weights for policy 1, policy_version 6910 (0.0007) -[2023-10-15 15:04:43,438][52833] Updated weights for policy 0, policy_version 6870 (0.0007) -[2023-10-15 15:04:43,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 14090240. Throughput: 0: 1790.9, 1: 1771.9. Samples: 3533578. Policy #0 lag: (min: 14.0, avg: 14.3, max: 26.0) -[2023-10-15 15:04:43,442][51532] Avg episode reward: [(0, '13.920'), (1, '12.830')] -[2023-10-15 15:04:43,811][52833] Updated weights for policy 0, policy_version 6880 (0.0007) -[2023-10-15 15:04:47,146][52866] Updated weights for policy 1, policy_version 6920 (0.0008) -[2023-10-15 15:04:47,465][52833] Updated weights for policy 0, policy_version 6890 (0.0008) -[2023-10-15 15:04:47,520][52866] Updated weights for policy 1, policy_version 6930 (0.0008) -[2023-10-15 15:04:47,844][52833] Updated weights for policy 0, policy_version 6900 (0.0010) -[2023-10-15 15:04:47,877][52866] Updated weights for policy 1, policy_version 6940 (0.0008) -[2023-10-15 15:04:48,219][52833] Updated weights for policy 0, policy_version 6910 (0.0007) -[2023-10-15 15:04:48,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 14188544. Throughput: 0: 1765.4, 1: 1767.5. Samples: 3544406. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) -[2023-10-15 15:04:48,441][51532] Avg episode reward: [(0, '11.780'), (1, '13.390')] -[2023-10-15 15:04:51,712][52866] Updated weights for policy 1, policy_version 6950 (0.0008) -[2023-10-15 15:04:52,078][52866] Updated weights for policy 1, policy_version 6960 (0.0007) -[2023-10-15 15:04:52,099][52833] Updated weights for policy 0, policy_version 6920 (0.0007) -[2023-10-15 15:04:52,436][52866] Updated weights for policy 1, policy_version 6970 (0.0007) -[2023-10-15 15:04:52,476][52833] Updated weights for policy 0, policy_version 6930 (0.0009) -[2023-10-15 15:04:52,847][52833] Updated weights for policy 0, policy_version 6940 (0.0009) -[2023-10-15 15:04:53,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 14254080. Throughput: 0: 1798.0, 1: 1778.7. Samples: 3565978. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) -[2023-10-15 15:04:53,442][51532] Avg episode reward: [(0, '12.150'), (1, '13.950')] -[2023-10-15 15:04:56,255][52866] Updated weights for policy 1, policy_version 6980 (0.0008) -[2023-10-15 15:04:56,624][52866] Updated weights for policy 1, policy_version 6990 (0.0007) -[2023-10-15 15:04:56,731][52833] Updated weights for policy 0, policy_version 6950 (0.0009) -[2023-10-15 15:04:56,984][52866] Updated weights for policy 1, policy_version 7000 (0.0008) -[2023-10-15 15:04:57,098][52833] Updated weights for policy 0, policy_version 6960 (0.0007) -[2023-10-15 15:04:57,469][52833] Updated weights for policy 0, policy_version 6970 (0.0009) -[2023-10-15 15:04:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 14319616. Throughput: 0: 1765.5, 1: 1757.0. Samples: 3585850. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-15 15:04:58,442][51532] Avg episode reward: [(0, '12.510'), (1, '14.050')] -[2023-10-15 15:05:00,758][52866] Updated weights for policy 1, policy_version 7010 (0.0009) -[2023-10-15 15:05:01,125][52866] Updated weights for policy 1, policy_version 7020 (0.0009) -[2023-10-15 15:05:01,306][52833] Updated weights for policy 0, policy_version 6980 (0.0008) -[2023-10-15 15:05:01,488][52866] Updated weights for policy 1, policy_version 7030 (0.0009) -[2023-10-15 15:05:01,670][52833] Updated weights for policy 0, policy_version 6990 (0.0007) -[2023-10-15 15:05:01,851][52866] Updated weights for policy 1, policy_version 7040 (0.0008) -[2023-10-15 15:05:02,046][52833] Updated weights for policy 0, policy_version 7000 (0.0007) -[2023-10-15 15:05:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 14385152. Throughput: 0: 1792.1, 1: 1781.7. Samples: 3597940. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-15 15:05:03,442][51532] Avg episode reward: [(0, '12.620'), (1, '12.690')] -[2023-10-15 15:05:05,625][52833] Updated weights for policy 0, policy_version 7010 (0.0007) -[2023-10-15 15:05:05,748][52866] Updated weights for policy 1, policy_version 7050 (0.0007) -[2023-10-15 15:05:05,996][52833] Updated weights for policy 0, policy_version 7020 (0.0007) -[2023-10-15 15:05:06,118][52866] Updated weights for policy 1, policy_version 7060 (0.0010) -[2023-10-15 15:05:06,364][52833] Updated weights for policy 0, policy_version 7030 (0.0008) -[2023-10-15 15:05:06,479][52866] Updated weights for policy 1, policy_version 7070 (0.0007) -[2023-10-15 15:05:06,738][52833] Updated weights for policy 0, policy_version 7040 (0.0007) -[2023-10-15 15:05:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 14450688. Throughput: 0: 1768.6, 1: 1764.4. Samples: 3617884. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 15:05:08,442][51532] Avg episode reward: [(0, '12.520'), (1, '12.400')] -[2023-10-15 15:05:10,402][52866] Updated weights for policy 1, policy_version 7080 (0.0008) -[2023-10-15 15:05:10,619][52833] Updated weights for policy 0, policy_version 7050 (0.0008) -[2023-10-15 15:05:10,778][52866] Updated weights for policy 1, policy_version 7090 (0.0009) -[2023-10-15 15:05:10,984][52833] Updated weights for policy 0, policy_version 7060 (0.0008) -[2023-10-15 15:05:11,137][52866] Updated weights for policy 1, policy_version 7100 (0.0007) -[2023-10-15 15:05:11,352][52833] Updated weights for policy 0, policy_version 7070 (0.0010) -[2023-10-15 15:05:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 14516224. Throughput: 0: 1767.0, 1: 1765.8. Samples: 3639664. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 15:05:13,442][51532] Avg episode reward: [(0, '14.540'), (1, '13.710')] -[2023-10-15 15:05:13,453][52410] Saving new best policy, reward=14.540! -[2023-10-15 15:05:14,941][52866] Updated weights for policy 1, policy_version 7110 (0.0007) -[2023-10-15 15:05:15,234][52833] Updated weights for policy 0, policy_version 7080 (0.0008) -[2023-10-15 15:05:15,316][52866] Updated weights for policy 1, policy_version 7120 (0.0007) -[2023-10-15 15:05:15,612][52833] Updated weights for policy 0, policy_version 7090 (0.0008) -[2023-10-15 15:05:15,674][52866] Updated weights for policy 1, policy_version 7130 (0.0008) -[2023-10-15 15:05:15,977][52833] Updated weights for policy 0, policy_version 7100 (0.0008) -[2023-10-15 15:05:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 14581760. Throughput: 0: 1776.2, 1: 1766.7. Samples: 3649674. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) -[2023-10-15 15:05:18,442][51532] Avg episode reward: [(0, '12.920'), (1, '12.960')] -[2023-10-15 15:05:19,437][52866] Updated weights for policy 1, policy_version 7140 (0.0008) -[2023-10-15 15:05:19,788][52833] Updated weights for policy 0, policy_version 7110 (0.0009) -[2023-10-15 15:05:19,811][52866] Updated weights for policy 1, policy_version 7150 (0.0009) -[2023-10-15 15:05:20,154][52833] Updated weights for policy 0, policy_version 7120 (0.0009) -[2023-10-15 15:05:20,174][52866] Updated weights for policy 1, policy_version 7160 (0.0008) -[2023-10-15 15:05:20,522][52833] Updated weights for policy 0, policy_version 7130 (0.0009) -[2023-10-15 15:05:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 14647296. Throughput: 0: 1765.2, 1: 1771.8. Samples: 3671536. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) -[2023-10-15 15:05:23,442][51532] Avg episode reward: [(0, '12.900'), (1, '13.800')] -[2023-10-15 15:05:23,927][52866] Updated weights for policy 1, policy_version 7170 (0.0007) -[2023-10-15 15:05:24,304][52866] Updated weights for policy 1, policy_version 7180 (0.0010) -[2023-10-15 15:05:24,489][52833] Updated weights for policy 0, policy_version 7140 (0.0008) -[2023-10-15 15:05:24,672][52866] Updated weights for policy 1, policy_version 7190 (0.0009) -[2023-10-15 15:05:24,861][52833] Updated weights for policy 0, policy_version 7150 (0.0009) -[2023-10-15 15:05:25,040][52866] Updated weights for policy 1, policy_version 7200 (0.0010) -[2023-10-15 15:05:25,220][52833] Updated weights for policy 0, policy_version 7160 (0.0008) -[2023-10-15 15:05:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 14712832. Throughput: 0: 1768.3, 1: 1785.9. Samples: 3693516. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) -[2023-10-15 15:05:28,442][51532] Avg episode reward: [(0, '12.700'), (1, '14.750')] -[2023-10-15 15:05:28,826][52866] Updated weights for policy 1, policy_version 7210 (0.0007) -[2023-10-15 15:05:28,887][52833] Updated weights for policy 0, policy_version 7170 (0.0010) -[2023-10-15 15:05:29,186][52866] Updated weights for policy 1, policy_version 7220 (0.0008) -[2023-10-15 15:05:29,253][52833] Updated weights for policy 0, policy_version 7180 (0.0008) -[2023-10-15 15:05:29,551][52866] Updated weights for policy 1, policy_version 7230 (0.0009) -[2023-10-15 15:05:29,620][52518] Saving new best policy, reward=14.750! -[2023-10-15 15:05:29,632][52833] Updated weights for policy 0, policy_version 7190 (0.0008) -[2023-10-15 15:05:29,995][52833] Updated weights for policy 0, policy_version 7200 (0.0008) -[2023-10-15 15:05:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 14778368. Throughput: 0: 1764.2, 1: 1764.2. Samples: 3703182. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) -[2023-10-15 15:05:33,442][51532] Avg episode reward: [(0, '13.970'), (1, '13.970')] -[2023-10-15 15:05:33,527][52866] Updated weights for policy 1, policy_version 7240 (0.0009) -[2023-10-15 15:05:33,809][52833] Updated weights for policy 0, policy_version 7210 (0.0008) -[2023-10-15 15:05:33,899][52866] Updated weights for policy 1, policy_version 7250 (0.0008) -[2023-10-15 15:05:34,177][52833] Updated weights for policy 0, policy_version 7220 (0.0008) -[2023-10-15 15:05:34,261][52866] Updated weights for policy 1, policy_version 7260 (0.0007) -[2023-10-15 15:05:34,548][52833] Updated weights for policy 0, policy_version 7230 (0.0007) -[2023-10-15 15:05:38,165][52866] Updated weights for policy 1, policy_version 7270 (0.0009) -[2023-10-15 15:05:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13995.8). Total num frames: 14843904. Throughput: 0: 1765.9, 1: 1774.0. Samples: 3725272. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:05:38,441][51532] Avg episode reward: [(0, '12.400'), (1, '14.820')] -[2023-10-15 15:05:38,472][52833] Updated weights for policy 0, policy_version 7240 (0.0007) -[2023-10-15 15:05:38,542][52866] Updated weights for policy 1, policy_version 7280 (0.0008) -[2023-10-15 15:05:38,843][52833] Updated weights for policy 0, policy_version 7250 (0.0007) -[2023-10-15 15:05:38,909][52866] Updated weights for policy 1, policy_version 7290 (0.0009) -[2023-10-15 15:05:39,131][52518] Saving new best policy, reward=14.820! -[2023-10-15 15:05:39,216][52833] Updated weights for policy 0, policy_version 7260 (0.0008) -[2023-10-15 15:05:42,783][52866] Updated weights for policy 1, policy_version 7300 (0.0008) -[2023-10-15 15:05:43,117][52833] Updated weights for policy 0, policy_version 7270 (0.0008) -[2023-10-15 15:05:43,148][52866] Updated weights for policy 1, policy_version 7310 (0.0007) -[2023-10-15 15:05:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13995.8). Total num frames: 14909440. Throughput: 0: 1789.9, 1: 1784.2. Samples: 3746684. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:05:43,443][51532] Avg episode reward: [(0, '12.440'), (1, '14.240')] -[2023-10-15 15:05:43,477][52833] Updated weights for policy 0, policy_version 7280 (0.0009) -[2023-10-15 15:05:43,520][52866] Updated weights for policy 1, policy_version 7320 (0.0008) -[2023-10-15 15:05:43,805][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000007328_7503872.pth... -[2023-10-15 15:05:43,838][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000005664_5799936.pth -[2023-10-15 15:05:43,844][52833] Updated weights for policy 0, policy_version 7290 (0.0009) -[2023-10-15 15:05:44,064][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000007296_7471104.pth... -[2023-10-15 15:05:44,105][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000005632_5767168.pth -[2023-10-15 15:05:47,248][52866] Updated weights for policy 1, policy_version 7330 (0.0008) -[2023-10-15 15:05:47,605][52866] Updated weights for policy 1, policy_version 7340 (0.0007) -[2023-10-15 15:05:47,784][52833] Updated weights for policy 0, policy_version 7300 (0.0010) -[2023-10-15 15:05:47,978][52866] Updated weights for policy 1, policy_version 7350 (0.0009) -[2023-10-15 15:05:48,150][52833] Updated weights for policy 0, policy_version 7310 (0.0008) -[2023-10-15 15:05:48,343][52866] Updated weights for policy 1, policy_version 7360 (0.0010) -[2023-10-15 15:05:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 14106.9). Total num frames: 15007744. Throughput: 0: 1758.6, 1: 1769.5. Samples: 3756706. Policy #0 lag: (min: 1.0, avg: 11.5, max: 33.0) -[2023-10-15 15:05:48,441][51532] Avg episode reward: [(0, '13.340'), (1, '13.890')] -[2023-10-15 15:05:48,517][52833] Updated weights for policy 0, policy_version 7320 (0.0009) -[2023-10-15 15:05:52,226][52833] Updated weights for policy 0, policy_version 7330 (0.0010) -[2023-10-15 15:05:52,261][52866] Updated weights for policy 1, policy_version 7370 (0.0008) -[2023-10-15 15:05:52,586][52833] Updated weights for policy 0, policy_version 7340 (0.0007) -[2023-10-15 15:05:52,629][52866] Updated weights for policy 1, policy_version 7380 (0.0007) -[2023-10-15 15:05:52,963][52833] Updated weights for policy 0, policy_version 7350 (0.0008) -[2023-10-15 15:05:53,001][52866] Updated weights for policy 1, policy_version 7390 (0.0010) -[2023-10-15 15:05:53,327][52833] Updated weights for policy 0, policy_version 7360 (0.0007) -[2023-10-15 15:05:53,441][51532] Fps is (10 sec: 19661.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 15106048. Throughput: 0: 1783.6, 1: 1782.0. Samples: 3778336. Policy #0 lag: (min: 13.0, avg: 19.0, max: 45.0) -[2023-10-15 15:05:53,442][51532] Avg episode reward: [(0, '12.230'), (1, '13.860')] -[2023-10-15 15:05:56,799][52866] Updated weights for policy 1, policy_version 7400 (0.0008) -[2023-10-15 15:05:57,168][52866] Updated weights for policy 1, policy_version 7410 (0.0008) -[2023-10-15 15:05:57,278][52833] Updated weights for policy 0, policy_version 7370 (0.0008) -[2023-10-15 15:05:57,531][52866] Updated weights for policy 1, policy_version 7420 (0.0007) -[2023-10-15 15:05:57,646][52833] Updated weights for policy 0, policy_version 7380 (0.0007) -[2023-10-15 15:05:58,006][52833] Updated weights for policy 0, policy_version 7390 (0.0007) -[2023-10-15 15:05:58,441][51532] Fps is (10 sec: 16383.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 15171584. Throughput: 0: 1759.4, 1: 1758.9. Samples: 3797986. Policy #0 lag: (min: 13.0, avg: 19.0, max: 45.0) -[2023-10-15 15:05:58,443][51532] Avg episode reward: [(0, '11.960'), (1, '13.640')] -[2023-10-15 15:06:01,179][52866] Updated weights for policy 1, policy_version 7430 (0.0008) -[2023-10-15 15:06:01,540][52866] Updated weights for policy 1, policy_version 7440 (0.0009) -[2023-10-15 15:06:01,821][52833] Updated weights for policy 0, policy_version 7400 (0.0007) -[2023-10-15 15:06:01,906][52866] Updated weights for policy 1, policy_version 7450 (0.0010) -[2023-10-15 15:06:02,187][52833] Updated weights for policy 0, policy_version 7410 (0.0007) -[2023-10-15 15:06:02,557][52833] Updated weights for policy 0, policy_version 7420 (0.0007) -[2023-10-15 15:06:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 15237120. Throughput: 0: 1777.7, 1: 1792.0. Samples: 3810310. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:06:03,442][51532] Avg episode reward: [(0, '12.830'), (1, '14.300')] -[2023-10-15 15:06:05,578][52866] Updated weights for policy 1, policy_version 7460 (0.0007) -[2023-10-15 15:06:05,955][52866] Updated weights for policy 1, policy_version 7470 (0.0007) -[2023-10-15 15:06:06,318][52866] Updated weights for policy 1, policy_version 7480 (0.0009) -[2023-10-15 15:06:06,471][52833] Updated weights for policy 0, policy_version 7430 (0.0008) -[2023-10-15 15:06:06,866][52833] Updated weights for policy 0, policy_version 7440 (0.0007) -[2023-10-15 15:06:07,248][52833] Updated weights for policy 0, policy_version 7450 (0.0008) -[2023-10-15 15:06:08,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 15302656. Throughput: 0: 1766.2, 1: 1760.8. Samples: 3830252. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:06:08,442][51532] Avg episode reward: [(0, '11.450'), (1, '14.180')] -[2023-10-15 15:06:10,155][52866] Updated weights for policy 1, policy_version 7490 (0.0007) -[2023-10-15 15:06:10,526][52866] Updated weights for policy 1, policy_version 7500 (0.0008) -[2023-10-15 15:06:10,896][52866] Updated weights for policy 1, policy_version 7510 (0.0008) -[2023-10-15 15:06:10,938][52833] Updated weights for policy 0, policy_version 7460 (0.0009) -[2023-10-15 15:06:11,258][52866] Updated weights for policy 1, policy_version 7520 (0.0008) -[2023-10-15 15:06:11,309][52833] Updated weights for policy 0, policy_version 7470 (0.0009) -[2023-10-15 15:06:11,666][52833] Updated weights for policy 0, policy_version 7480 (0.0008) -[2023-10-15 15:06:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 15368192. Throughput: 0: 1749.6, 1: 1765.3. Samples: 3851688. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-15 15:06:13,442][51532] Avg episode reward: [(0, '12.110'), (1, '14.470')] -[2023-10-15 15:06:15,027][52866] Updated weights for policy 1, policy_version 7530 (0.0010) -[2023-10-15 15:06:15,315][52833] Updated weights for policy 0, policy_version 7490 (0.0009) -[2023-10-15 15:06:15,395][52866] Updated weights for policy 1, policy_version 7540 (0.0008) -[2023-10-15 15:06:15,691][52833] Updated weights for policy 0, policy_version 7500 (0.0008) -[2023-10-15 15:06:15,765][52866] Updated weights for policy 1, policy_version 7550 (0.0009) -[2023-10-15 15:06:16,075][52833] Updated weights for policy 0, policy_version 7510 (0.0008) -[2023-10-15 15:06:16,437][52833] Updated weights for policy 0, policy_version 7520 (0.0008) -[2023-10-15 15:06:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 15433728. Throughput: 0: 1770.0, 1: 1766.2. Samples: 3862314. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-15 15:06:18,442][51532] Avg episode reward: [(0, '12.270'), (1, '15.740')] -[2023-10-15 15:06:18,443][52518] Saving new best policy, reward=15.740! -[2023-10-15 15:06:19,530][52866] Updated weights for policy 1, policy_version 7560 (0.0009) -[2023-10-15 15:06:19,895][52866] Updated weights for policy 1, policy_version 7570 (0.0009) -[2023-10-15 15:06:20,257][52866] Updated weights for policy 1, policy_version 7580 (0.0009) -[2023-10-15 15:06:20,272][52833] Updated weights for policy 0, policy_version 7530 (0.0007) -[2023-10-15 15:06:20,643][52833] Updated weights for policy 0, policy_version 7540 (0.0007) -[2023-10-15 15:06:21,005][52833] Updated weights for policy 0, policy_version 7550 (0.0007) -[2023-10-15 15:06:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 15499264. Throughput: 0: 1754.1, 1: 1771.1. Samples: 3883910. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) -[2023-10-15 15:06:23,442][51532] Avg episode reward: [(0, '12.470'), (1, '16.090')] -[2023-10-15 15:06:23,443][52518] Saving new best policy, reward=16.090! -[2023-10-15 15:06:24,112][52866] Updated weights for policy 1, policy_version 7590 (0.0007) -[2023-10-15 15:06:24,488][52866] Updated weights for policy 1, policy_version 7600 (0.0007) -[2023-10-15 15:06:24,780][52833] Updated weights for policy 0, policy_version 7560 (0.0009) -[2023-10-15 15:06:24,855][52866] Updated weights for policy 1, policy_version 7610 (0.0010) -[2023-10-15 15:06:25,150][52833] Updated weights for policy 0, policy_version 7570 (0.0009) -[2023-10-15 15:06:25,518][52833] Updated weights for policy 0, policy_version 7580 (0.0011) -[2023-10-15 15:06:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 15564800. Throughput: 0: 1760.6, 1: 1784.6. Samples: 3906216. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) -[2023-10-15 15:06:28,441][51532] Avg episode reward: [(0, '12.410'), (1, '16.160')] -[2023-10-15 15:06:28,512][52866] Updated weights for policy 1, policy_version 7620 (0.0009) -[2023-10-15 15:06:28,874][52866] Updated weights for policy 1, policy_version 7630 (0.0010) -[2023-10-15 15:06:29,237][52866] Updated weights for policy 1, policy_version 7640 (0.0009) -[2023-10-15 15:06:29,340][52833] Updated weights for policy 0, policy_version 7590 (0.0009) -[2023-10-15 15:06:29,528][52518] Saving new best policy, reward=16.160! -[2023-10-15 15:06:29,705][52833] Updated weights for policy 0, policy_version 7600 (0.0009) -[2023-10-15 15:06:30,078][52833] Updated weights for policy 0, policy_version 7610 (0.0010) -[2023-10-15 15:06:33,148][52866] Updated weights for policy 1, policy_version 7650 (0.0008) -[2023-10-15 15:06:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 15630336. Throughput: 0: 1761.2, 1: 1776.1. Samples: 3915886. Policy #0 lag: (min: 28.0, avg: 30.4, max: 60.0) -[2023-10-15 15:06:33,441][51532] Avg episode reward: [(0, '12.080'), (1, '16.690')] -[2023-10-15 15:06:33,520][52866] Updated weights for policy 1, policy_version 7660 (0.0010) -[2023-10-15 15:06:33,886][52866] Updated weights for policy 1, policy_version 7670 (0.0008) -[2023-10-15 15:06:33,939][52833] Updated weights for policy 0, policy_version 7620 (0.0008) -[2023-10-15 15:06:34,245][52518] Saving new best policy, reward=16.690! -[2023-10-15 15:06:34,247][52866] Updated weights for policy 1, policy_version 7680 (0.0007) -[2023-10-15 15:06:34,308][52833] Updated weights for policy 0, policy_version 7630 (0.0007) -[2023-10-15 15:06:34,676][52833] Updated weights for policy 0, policy_version 7640 (0.0009) -[2023-10-15 15:06:37,965][52866] Updated weights for policy 1, policy_version 7690 (0.0008) -[2023-10-15 15:06:38,306][52833] Updated weights for policy 0, policy_version 7650 (0.0009) -[2023-10-15 15:06:38,329][52866] Updated weights for policy 1, policy_version 7700 (0.0008) -[2023-10-15 15:06:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 15695872. Throughput: 0: 1763.8, 1: 1781.8. Samples: 3937888. Policy #0 lag: (min: 28.0, avg: 30.4, max: 60.0) -[2023-10-15 15:06:38,441][51532] Avg episode reward: [(0, '13.210'), (1, '16.010')] -[2023-10-15 15:06:38,666][52833] Updated weights for policy 0, policy_version 7660 (0.0008) -[2023-10-15 15:06:38,690][52866] Updated weights for policy 1, policy_version 7710 (0.0007) -[2023-10-15 15:06:39,035][52833] Updated weights for policy 0, policy_version 7670 (0.0008) -[2023-10-15 15:06:39,409][52833] Updated weights for policy 0, policy_version 7680 (0.0008) -[2023-10-15 15:06:42,568][52866] Updated weights for policy 1, policy_version 7720 (0.0007) -[2023-10-15 15:06:42,941][52866] Updated weights for policy 1, policy_version 7730 (0.0008) -[2023-10-15 15:06:43,254][52833] Updated weights for policy 0, policy_version 7690 (0.0007) -[2023-10-15 15:06:43,307][52866] Updated weights for policy 1, policy_version 7740 (0.0009) -[2023-10-15 15:06:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 13995.8). Total num frames: 15761408. Throughput: 0: 1796.0, 1: 1791.4. Samples: 3959420. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) -[2023-10-15 15:06:43,442][51532] Avg episode reward: [(0, '12.150'), (1, '15.460')] -[2023-10-15 15:06:43,621][52833] Updated weights for policy 0, policy_version 7700 (0.0007) -[2023-10-15 15:06:43,986][52833] Updated weights for policy 0, policy_version 7710 (0.0008) -[2023-10-15 15:06:47,004][52866] Updated weights for policy 1, policy_version 7750 (0.0008) -[2023-10-15 15:06:47,362][52866] Updated weights for policy 1, policy_version 7760 (0.0007) -[2023-10-15 15:06:47,723][52866] Updated weights for policy 1, policy_version 7770 (0.0008) -[2023-10-15 15:06:47,812][52833] Updated weights for policy 0, policy_version 7720 (0.0008) -[2023-10-15 15:06:48,189][52833] Updated weights for policy 0, policy_version 7730 (0.0008) -[2023-10-15 15:06:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 15859712. Throughput: 0: 1774.5, 1: 1782.1. Samples: 3970356. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) -[2023-10-15 15:06:48,441][51532] Avg episode reward: [(0, '13.560'), (1, '15.400')] -[2023-10-15 15:06:48,564][52833] Updated weights for policy 0, policy_version 7740 (0.0009) -[2023-10-15 15:06:51,436][52866] Updated weights for policy 1, policy_version 7780 (0.0009) -[2023-10-15 15:06:51,804][52866] Updated weights for policy 1, policy_version 7790 (0.0007) -[2023-10-15 15:06:52,164][52866] Updated weights for policy 1, policy_version 7800 (0.0009) -[2023-10-15 15:06:52,392][52833] Updated weights for policy 0, policy_version 7750 (0.0009) -[2023-10-15 15:06:52,771][52833] Updated weights for policy 0, policy_version 7760 (0.0009) -[2023-10-15 15:06:53,144][52833] Updated weights for policy 0, policy_version 7770 (0.0011) -[2023-10-15 15:06:53,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 15958016. Throughput: 0: 1797.6, 1: 1794.4. Samples: 3991892. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-15 15:06:53,442][51532] Avg episode reward: [(0, '14.200'), (1, '14.790')] -[2023-10-15 15:06:56,000][52866] Updated weights for policy 1, policy_version 7810 (0.0009) -[2023-10-15 15:06:56,360][52866] Updated weights for policy 1, policy_version 7820 (0.0008) -[2023-10-15 15:06:56,729][52866] Updated weights for policy 1, policy_version 7830 (0.0009) -[2023-10-15 15:06:56,945][52833] Updated weights for policy 0, policy_version 7780 (0.0009) -[2023-10-15 15:06:57,095][52866] Updated weights for policy 1, policy_version 7840 (0.0008) -[2023-10-15 15:06:57,305][52833] Updated weights for policy 0, policy_version 7790 (0.0007) -[2023-10-15 15:06:57,672][52833] Updated weights for policy 0, policy_version 7800 (0.0007) -[2023-10-15 15:06:58,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16023552. Throughput: 0: 1784.8, 1: 1783.2. Samples: 4012246. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) -[2023-10-15 15:06:58,442][51532] Avg episode reward: [(0, '15.000'), (1, '15.190')] -[2023-10-15 15:06:58,453][52410] Saving new best policy, reward=15.000! -[2023-10-15 15:07:00,749][52866] Updated weights for policy 1, policy_version 7850 (0.0008) -[2023-10-15 15:07:01,121][52866] Updated weights for policy 1, policy_version 7860 (0.0007) -[2023-10-15 15:07:01,334][52833] Updated weights for policy 0, policy_version 7810 (0.0007) -[2023-10-15 15:07:01,489][52866] Updated weights for policy 1, policy_version 7870 (0.0007) -[2023-10-15 15:07:01,697][52833] Updated weights for policy 0, policy_version 7820 (0.0007) -[2023-10-15 15:07:02,064][52833] Updated weights for policy 0, policy_version 7830 (0.0007) -[2023-10-15 15:07:02,442][52833] Updated weights for policy 0, policy_version 7840 (0.0008) -[2023-10-15 15:07:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16089088. Throughput: 0: 1791.8, 1: 1801.9. Samples: 4024032. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) -[2023-10-15 15:07:03,442][51532] Avg episode reward: [(0, '16.390'), (1, '14.690')] -[2023-10-15 15:07:03,442][52410] Saving new best policy, reward=16.390! -[2023-10-15 15:07:05,289][52866] Updated weights for policy 1, policy_version 7880 (0.0010) -[2023-10-15 15:07:05,654][52866] Updated weights for policy 1, policy_version 7890 (0.0010) -[2023-10-15 15:07:06,021][52866] Updated weights for policy 1, policy_version 7900 (0.0009) -[2023-10-15 15:07:06,262][52833] Updated weights for policy 0, policy_version 7850 (0.0009) -[2023-10-15 15:07:06,623][52833] Updated weights for policy 0, policy_version 7860 (0.0009) -[2023-10-15 15:07:06,997][52833] Updated weights for policy 0, policy_version 7870 (0.0009) -[2023-10-15 15:07:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16154624. Throughput: 0: 1785.6, 1: 1786.4. Samples: 4044646. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:07:08,442][51532] Avg episode reward: [(0, '16.130'), (1, '13.190')] -[2023-10-15 15:07:09,874][52866] Updated weights for policy 1, policy_version 7910 (0.0008) -[2023-10-15 15:07:10,244][52866] Updated weights for policy 1, policy_version 7920 (0.0009) -[2023-10-15 15:07:10,523][52833] Updated weights for policy 0, policy_version 7880 (0.0008) -[2023-10-15 15:07:10,608][52866] Updated weights for policy 1, policy_version 7930 (0.0007) -[2023-10-15 15:07:10,880][52833] Updated weights for policy 0, policy_version 7890 (0.0008) -[2023-10-15 15:07:11,253][52833] Updated weights for policy 0, policy_version 7900 (0.0010) -[2023-10-15 15:07:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16220160. Throughput: 0: 1783.5, 1: 1780.8. Samples: 4066610. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:07:13,442][51532] Avg episode reward: [(0, '17.290'), (1, '13.510')] -[2023-10-15 15:07:13,450][52410] Saving new best policy, reward=17.290! -[2023-10-15 15:07:14,538][52866] Updated weights for policy 1, policy_version 7940 (0.0008) -[2023-10-15 15:07:14,909][52866] Updated weights for policy 1, policy_version 7950 (0.0007) -[2023-10-15 15:07:15,246][52833] Updated weights for policy 0, policy_version 7910 (0.0010) -[2023-10-15 15:07:15,270][52866] Updated weights for policy 1, policy_version 7960 (0.0007) -[2023-10-15 15:07:15,613][52833] Updated weights for policy 0, policy_version 7920 (0.0008) -[2023-10-15 15:07:16,001][52833] Updated weights for policy 0, policy_version 7930 (0.0012) -[2023-10-15 15:07:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16285696. Throughput: 0: 1792.3, 1: 1781.6. Samples: 4076710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:07:18,442][51532] Avg episode reward: [(0, '16.530'), (1, '13.690')] -[2023-10-15 15:07:18,964][52866] Updated weights for policy 1, policy_version 7970 (0.0008) -[2023-10-15 15:07:19,329][52866] Updated weights for policy 1, policy_version 7980 (0.0007) -[2023-10-15 15:07:19,706][52866] Updated weights for policy 1, policy_version 7990 (0.0007) -[2023-10-15 15:07:19,808][52833] Updated weights for policy 0, policy_version 7940 (0.0009) -[2023-10-15 15:07:20,070][52866] Updated weights for policy 1, policy_version 8000 (0.0007) -[2023-10-15 15:07:20,182][52833] Updated weights for policy 0, policy_version 7950 (0.0007) -[2023-10-15 15:07:20,553][52833] Updated weights for policy 0, policy_version 7960 (0.0010) -[2023-10-15 15:07:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16351232. Throughput: 0: 1777.0, 1: 1789.0. Samples: 4098358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:07:23,442][51532] Avg episode reward: [(0, '15.980'), (1, '14.410')] -[2023-10-15 15:07:23,855][52866] Updated weights for policy 1, policy_version 8010 (0.0010) -[2023-10-15 15:07:24,219][52866] Updated weights for policy 1, policy_version 8020 (0.0009) -[2023-10-15 15:07:24,388][52833] Updated weights for policy 0, policy_version 7970 (0.0010) -[2023-10-15 15:07:24,580][52866] Updated weights for policy 1, policy_version 8030 (0.0007) -[2023-10-15 15:07:24,759][52833] Updated weights for policy 0, policy_version 7980 (0.0009) -[2023-10-15 15:07:25,139][52833] Updated weights for policy 0, policy_version 7990 (0.0009) -[2023-10-15 15:07:25,514][52833] Updated weights for policy 0, policy_version 8000 (0.0008) -[2023-10-15 15:07:28,355][52866] Updated weights for policy 1, policy_version 8040 (0.0007) -[2023-10-15 15:07:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16416768. Throughput: 0: 1768.8, 1: 1811.6. Samples: 4120538. Policy #0 lag: (min: 1.0, avg: 15.4, max: 33.0) -[2023-10-15 15:07:28,441][51532] Avg episode reward: [(0, '16.200'), (1, '13.910')] -[2023-10-15 15:07:28,732][52866] Updated weights for policy 1, policy_version 8050 (0.0009) -[2023-10-15 15:07:29,103][52866] Updated weights for policy 1, policy_version 8060 (0.0008) -[2023-10-15 15:07:29,336][52833] Updated weights for policy 0, policy_version 8010 (0.0009) -[2023-10-15 15:07:29,699][52833] Updated weights for policy 0, policy_version 8020 (0.0011) -[2023-10-15 15:07:30,070][52833] Updated weights for policy 0, policy_version 8030 (0.0010) -[2023-10-15 15:07:33,011][52866] Updated weights for policy 1, policy_version 8070 (0.0008) -[2023-10-15 15:07:33,369][52866] Updated weights for policy 1, policy_version 8080 (0.0009) -[2023-10-15 15:07:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 16482304. Throughput: 0: 1764.6, 1: 1781.8. Samples: 4129942. Policy #0 lag: (min: 1.0, avg: 15.4, max: 33.0) -[2023-10-15 15:07:33,442][51532] Avg episode reward: [(0, '15.330'), (1, '14.980')] -[2023-10-15 15:07:33,733][52866] Updated weights for policy 1, policy_version 8090 (0.0008) -[2023-10-15 15:07:34,114][52833] Updated weights for policy 0, policy_version 8040 (0.0008) -[2023-10-15 15:07:34,485][52833] Updated weights for policy 0, policy_version 8050 (0.0009) -[2023-10-15 15:07:34,858][52833] Updated weights for policy 0, policy_version 8060 (0.0009) -[2023-10-15 15:07:37,537][52866] Updated weights for policy 1, policy_version 8100 (0.0008) -[2023-10-15 15:07:37,904][52866] Updated weights for policy 1, policy_version 8110 (0.0007) -[2023-10-15 15:07:38,263][52866] Updated weights for policy 1, policy_version 8120 (0.0008) -[2023-10-15 15:07:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16547840. Throughput: 0: 1763.5, 1: 1798.3. Samples: 4152174. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 15:07:38,441][51532] Avg episode reward: [(0, '14.090'), (1, '14.270')] -[2023-10-15 15:07:38,556][52833] Updated weights for policy 0, policy_version 8070 (0.0007) -[2023-10-15 15:07:38,949][52833] Updated weights for policy 0, policy_version 8080 (0.0008) -[2023-10-15 15:07:39,323][52833] Updated weights for policy 0, policy_version 8090 (0.0009) -[2023-10-15 15:07:41,807][52866] Updated weights for policy 1, policy_version 8130 (0.0009) -[2023-10-15 15:07:42,163][52866] Updated weights for policy 1, policy_version 8140 (0.0011) -[2023-10-15 15:07:42,532][52866] Updated weights for policy 1, policy_version 8150 (0.0012) -[2023-10-15 15:07:42,898][52866] Updated weights for policy 1, policy_version 8160 (0.0011) -[2023-10-15 15:07:43,025][52833] Updated weights for policy 0, policy_version 8100 (0.0010) -[2023-10-15 15:07:43,393][52833] Updated weights for policy 0, policy_version 8110 (0.0010) -[2023-10-15 15:07:43,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14218.0). Total num frames: 16646144. Throughput: 0: 1794.2, 1: 1776.2. Samples: 4172914. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 15:07:43,441][51532] Avg episode reward: [(0, '13.250'), (1, '13.270')] -[2023-10-15 15:07:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000008160_8355840.pth... -[2023-10-15 15:07:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000006496_6651904.pth -[2023-10-15 15:07:43,491][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000008160_8355840.pth -[2023-10-15 15:07:43,766][52833] Updated weights for policy 0, policy_version 8120 (0.0008) -[2023-10-15 15:07:44,055][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000008128_8323072.pth... -[2023-10-15 15:07:44,084][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000006464_6619136.pth -[2023-10-15 15:07:44,088][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000008128_8323072.pth -[2023-10-15 15:07:46,648][52866] Updated weights for policy 1, policy_version 8170 (0.0009) -[2023-10-15 15:07:47,023][52866] Updated weights for policy 1, policy_version 8180 (0.0009) -[2023-10-15 15:07:47,396][52866] Updated weights for policy 1, policy_version 8190 (0.0008) -[2023-10-15 15:07:47,611][52833] Updated weights for policy 0, policy_version 8130 (0.0008) -[2023-10-15 15:07:47,984][52833] Updated weights for policy 0, policy_version 8140 (0.0009) -[2023-10-15 15:07:48,353][52833] Updated weights for policy 0, policy_version 8150 (0.0007) -[2023-10-15 15:07:48,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14199.4, 300 sec: 14106.9). Total num frames: 16711680. Throughput: 0: 1763.5, 1: 1791.5. Samples: 4184010. Policy #0 lag: (min: 2.0, avg: 17.8, max: 34.0) -[2023-10-15 15:07:48,442][51532] Avg episode reward: [(0, '13.500'), (1, '13.460')] -[2023-10-15 15:07:48,717][52833] Updated weights for policy 0, policy_version 8160 (0.0010) -[2023-10-15 15:07:51,190][52866] Updated weights for policy 1, policy_version 8200 (0.0007) -[2023-10-15 15:07:51,561][52866] Updated weights for policy 1, policy_version 8210 (0.0008) -[2023-10-15 15:07:51,933][52866] Updated weights for policy 1, policy_version 8220 (0.0008) -[2023-10-15 15:07:52,492][52833] Updated weights for policy 0, policy_version 8170 (0.0007) -[2023-10-15 15:07:52,853][52833] Updated weights for policy 0, policy_version 8180 (0.0008) -[2023-10-15 15:07:53,222][52833] Updated weights for policy 0, policy_version 8190 (0.0008) -[2023-10-15 15:07:53,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 16809984. Throughput: 0: 1785.4, 1: 1777.5. Samples: 4204974. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 15:07:53,442][51532] Avg episode reward: [(0, '13.330'), (1, '13.860')] -[2023-10-15 15:07:55,592][52866] Updated weights for policy 1, policy_version 8230 (0.0009) -[2023-10-15 15:07:55,972][52866] Updated weights for policy 1, policy_version 8240 (0.0009) -[2023-10-15 15:07:56,345][52866] Updated weights for policy 1, policy_version 8250 (0.0010) -[2023-10-15 15:07:56,870][52833] Updated weights for policy 0, policy_version 8200 (0.0008) -[2023-10-15 15:07:57,240][52833] Updated weights for policy 0, policy_version 8210 (0.0008) -[2023-10-15 15:07:57,609][52833] Updated weights for policy 0, policy_version 8220 (0.0007) -[2023-10-15 15:07:58,441][51532] Fps is (10 sec: 16384.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 16875520. Throughput: 0: 1759.4, 1: 1781.2. Samples: 4225936. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 15:07:58,441][51532] Avg episode reward: [(0, '14.520'), (1, '14.580')] -[2023-10-15 15:08:00,096][52866] Updated weights for policy 1, policy_version 8260 (0.0009) -[2023-10-15 15:08:00,473][52866] Updated weights for policy 1, policy_version 8270 (0.0011) -[2023-10-15 15:08:00,838][52866] Updated weights for policy 1, policy_version 8280 (0.0011) -[2023-10-15 15:08:01,369][52833] Updated weights for policy 0, policy_version 8230 (0.0008) -[2023-10-15 15:08:01,734][52833] Updated weights for policy 0, policy_version 8240 (0.0008) -[2023-10-15 15:08:02,109][52833] Updated weights for policy 0, policy_version 8250 (0.0008) -[2023-10-15 15:08:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 16941056. Throughput: 0: 1784.3, 1: 1786.0. Samples: 4237374. Policy #0 lag: (min: 30.0, avg: 36.6, max: 62.0) -[2023-10-15 15:08:03,442][51532] Avg episode reward: [(0, '13.670'), (1, '14.990')] -[2023-10-15 15:08:04,695][52866] Updated weights for policy 1, policy_version 8290 (0.0009) -[2023-10-15 15:08:05,068][52866] Updated weights for policy 1, policy_version 8300 (0.0009) -[2023-10-15 15:08:05,437][52866] Updated weights for policy 1, policy_version 8310 (0.0009) -[2023-10-15 15:08:05,800][52866] Updated weights for policy 1, policy_version 8320 (0.0009) -[2023-10-15 15:08:05,920][52833] Updated weights for policy 0, policy_version 8260 (0.0009) -[2023-10-15 15:08:06,281][52833] Updated weights for policy 0, policy_version 8270 (0.0007) -[2023-10-15 15:08:06,653][52833] Updated weights for policy 0, policy_version 8280 (0.0007) -[2023-10-15 15:08:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17006592. Throughput: 0: 1775.8, 1: 1774.2. Samples: 4258106. Policy #0 lag: (min: 30.0, avg: 36.6, max: 62.0) -[2023-10-15 15:08:08,442][51532] Avg episode reward: [(0, '13.650'), (1, '15.880')] -[2023-10-15 15:08:09,699][52866] Updated weights for policy 1, policy_version 8330 (0.0010) -[2023-10-15 15:08:10,079][52866] Updated weights for policy 1, policy_version 8340 (0.0012) -[2023-10-15 15:08:10,444][52866] Updated weights for policy 1, policy_version 8350 (0.0008) -[2023-10-15 15:08:10,464][52833] Updated weights for policy 0, policy_version 8290 (0.0007) -[2023-10-15 15:08:10,831][52833] Updated weights for policy 0, policy_version 8300 (0.0008) -[2023-10-15 15:08:11,205][52833] Updated weights for policy 0, policy_version 8310 (0.0008) -[2023-10-15 15:08:11,576][52833] Updated weights for policy 0, policy_version 8320 (0.0009) -[2023-10-15 15:08:13,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17072128. Throughput: 0: 1776.7, 1: 1767.1. Samples: 4280010. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) -[2023-10-15 15:08:13,441][51532] Avg episode reward: [(0, '13.860'), (1, '15.970')] -[2023-10-15 15:08:14,346][52866] Updated weights for policy 1, policy_version 8360 (0.0007) -[2023-10-15 15:08:14,719][52866] Updated weights for policy 1, policy_version 8370 (0.0007) -[2023-10-15 15:08:15,077][52866] Updated weights for policy 1, policy_version 8380 (0.0007) -[2023-10-15 15:08:15,257][52833] Updated weights for policy 0, policy_version 8330 (0.0008) -[2023-10-15 15:08:15,617][52833] Updated weights for policy 0, policy_version 8340 (0.0008) -[2023-10-15 15:08:15,988][52833] Updated weights for policy 0, policy_version 8350 (0.0008) -[2023-10-15 15:08:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17137664. Throughput: 0: 1787.3, 1: 1772.2. Samples: 4290122. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) -[2023-10-15 15:08:18,441][51532] Avg episode reward: [(0, '13.250'), (1, '16.260')] -[2023-10-15 15:08:18,818][52866] Updated weights for policy 1, policy_version 8390 (0.0010) -[2023-10-15 15:08:19,186][52866] Updated weights for policy 1, policy_version 8400 (0.0010) -[2023-10-15 15:08:19,544][52866] Updated weights for policy 1, policy_version 8410 (0.0010) -[2023-10-15 15:08:19,797][52833] Updated weights for policy 0, policy_version 8360 (0.0009) -[2023-10-15 15:08:20,166][52833] Updated weights for policy 0, policy_version 8370 (0.0009) -[2023-10-15 15:08:20,545][52833] Updated weights for policy 0, policy_version 8380 (0.0008) -[2023-10-15 15:08:23,275][52866] Updated weights for policy 1, policy_version 8420 (0.0008) -[2023-10-15 15:08:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17203200. Throughput: 0: 1774.9, 1: 1772.6. Samples: 4311812. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) -[2023-10-15 15:08:23,442][51532] Avg episode reward: [(0, '14.500'), (1, '16.300')] -[2023-10-15 15:08:23,661][52866] Updated weights for policy 1, policy_version 8430 (0.0010) -[2023-10-15 15:08:24,023][52866] Updated weights for policy 1, policy_version 8440 (0.0011) -[2023-10-15 15:08:24,419][52833] Updated weights for policy 0, policy_version 8390 (0.0009) -[2023-10-15 15:08:24,810][52833] Updated weights for policy 0, policy_version 8400 (0.0009) -[2023-10-15 15:08:25,183][52833] Updated weights for policy 0, policy_version 8410 (0.0008) -[2023-10-15 15:08:27,790][52866] Updated weights for policy 1, policy_version 8450 (0.0009) -[2023-10-15 15:08:28,152][52866] Updated weights for policy 1, policy_version 8460 (0.0009) -[2023-10-15 15:08:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17268736. Throughput: 0: 1773.9, 1: 1797.7. Samples: 4333634. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) -[2023-10-15 15:08:28,441][51532] Avg episode reward: [(0, '14.280'), (1, '16.400')] -[2023-10-15 15:08:28,518][52866] Updated weights for policy 1, policy_version 8470 (0.0009) -[2023-10-15 15:08:28,888][52866] Updated weights for policy 1, policy_version 8480 (0.0007) -[2023-10-15 15:08:28,929][52833] Updated weights for policy 0, policy_version 8420 (0.0009) -[2023-10-15 15:08:29,293][52833] Updated weights for policy 0, policy_version 8430 (0.0009) -[2023-10-15 15:08:29,671][52833] Updated weights for policy 0, policy_version 8440 (0.0009) -[2023-10-15 15:08:32,803][52866] Updated weights for policy 1, policy_version 8490 (0.0010) -[2023-10-15 15:08:33,168][52866] Updated weights for policy 1, policy_version 8500 (0.0008) -[2023-10-15 15:08:33,433][52833] Updated weights for policy 0, policy_version 8450 (0.0009) -[2023-10-15 15:08:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17334272. Throughput: 0: 1774.7, 1: 1771.9. Samples: 4343606. Policy #0 lag: (min: 21.0, avg: 21.2, max: 31.0) -[2023-10-15 15:08:33,441][51532] Avg episode reward: [(0, '15.550'), (1, '16.140')] -[2023-10-15 15:08:33,535][52866] Updated weights for policy 1, policy_version 8510 (0.0007) -[2023-10-15 15:08:33,799][52833] Updated weights for policy 0, policy_version 8460 (0.0008) -[2023-10-15 15:08:34,170][52833] Updated weights for policy 0, policy_version 8470 (0.0008) -[2023-10-15 15:08:34,547][52833] Updated weights for policy 0, policy_version 8480 (0.0008) -[2023-10-15 15:08:37,454][52866] Updated weights for policy 1, policy_version 8520 (0.0007) -[2023-10-15 15:08:37,821][52866] Updated weights for policy 1, policy_version 8530 (0.0009) -[2023-10-15 15:08:38,186][52866] Updated weights for policy 1, policy_version 8540 (0.0008) -[2023-10-15 15:08:38,255][52833] Updated weights for policy 0, policy_version 8490 (0.0007) -[2023-10-15 15:08:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14218.0). Total num frames: 17432576. Throughput: 0: 1774.9, 1: 1799.7. Samples: 4365830. Policy #0 lag: (min: 21.0, avg: 21.2, max: 31.0) -[2023-10-15 15:08:38,441][51532] Avg episode reward: [(0, '14.710'), (1, '15.340')] -[2023-10-15 15:08:38,625][52833] Updated weights for policy 0, policy_version 8500 (0.0010) -[2023-10-15 15:08:38,993][52833] Updated weights for policy 0, policy_version 8510 (0.0008) -[2023-10-15 15:08:41,908][52866] Updated weights for policy 1, policy_version 8550 (0.0009) -[2023-10-15 15:08:42,267][52866] Updated weights for policy 1, policy_version 8560 (0.0009) -[2023-10-15 15:08:42,638][52866] Updated weights for policy 1, policy_version 8570 (0.0008) -[2023-10-15 15:08:42,862][52833] Updated weights for policy 0, policy_version 8520 (0.0008) -[2023-10-15 15:08:43,230][52833] Updated weights for policy 0, policy_version 8530 (0.0008) -[2023-10-15 15:08:43,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14106.9). Total num frames: 17498112. Throughput: 0: 1797.3, 1: 1771.6. Samples: 4386534. Policy #0 lag: (min: 26.0, avg: 27.2, max: 50.0) -[2023-10-15 15:08:43,441][51532] Avg episode reward: [(0, '14.900'), (1, '16.680')] -[2023-10-15 15:08:43,605][52833] Updated weights for policy 0, policy_version 8540 (0.0008) -[2023-10-15 15:08:46,459][52866] Updated weights for policy 1, policy_version 8580 (0.0008) -[2023-10-15 15:08:46,829][52866] Updated weights for policy 1, policy_version 8590 (0.0011) -[2023-10-15 15:08:47,200][52866] Updated weights for policy 1, policy_version 8600 (0.0010) -[2023-10-15 15:08:47,376][52833] Updated weights for policy 0, policy_version 8550 (0.0009) -[2023-10-15 15:08:47,748][52833] Updated weights for policy 0, policy_version 8560 (0.0008) -[2023-10-15 15:08:48,121][52833] Updated weights for policy 0, policy_version 8570 (0.0007) -[2023-10-15 15:08:48,443][51532] Fps is (10 sec: 16380.3, 60 sec: 14745.1, 300 sec: 14217.9). Total num frames: 17596416. Throughput: 0: 1773.3, 1: 1795.7. Samples: 4397988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:08:48,444][51532] Avg episode reward: [(0, '14.620'), (1, '17.000')] -[2023-10-15 15:08:48,445][52518] Saving new best policy, reward=17.000! -[2023-10-15 15:08:51,127][52866] Updated weights for policy 1, policy_version 8610 (0.0009) -[2023-10-15 15:08:51,485][52866] Updated weights for policy 1, policy_version 8620 (0.0009) -[2023-10-15 15:08:51,855][52866] Updated weights for policy 1, policy_version 8630 (0.0008) -[2023-10-15 15:08:51,948][52833] Updated weights for policy 0, policy_version 8580 (0.0008) -[2023-10-15 15:08:52,220][52866] Updated weights for policy 1, policy_version 8640 (0.0007) -[2023-10-15 15:08:52,316][52833] Updated weights for policy 0, policy_version 8590 (0.0010) -[2023-10-15 15:08:52,689][52833] Updated weights for policy 0, policy_version 8600 (0.0008) -[2023-10-15 15:08:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17661952. Throughput: 0: 1791.7, 1: 1775.0. Samples: 4418608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:08:53,442][51532] Avg episode reward: [(0, '15.290'), (1, '16.150')] -[2023-10-15 15:08:55,963][52866] Updated weights for policy 1, policy_version 8650 (0.0008) -[2023-10-15 15:08:56,324][52866] Updated weights for policy 1, policy_version 8660 (0.0008) -[2023-10-15 15:08:56,506][52833] Updated weights for policy 0, policy_version 8610 (0.0008) -[2023-10-15 15:08:56,694][52866] Updated weights for policy 1, policy_version 8670 (0.0007) -[2023-10-15 15:08:56,875][52833] Updated weights for policy 0, policy_version 8620 (0.0008) -[2023-10-15 15:08:57,243][52833] Updated weights for policy 0, policy_version 8630 (0.0009) -[2023-10-15 15:08:57,615][52833] Updated weights for policy 0, policy_version 8640 (0.0010) -[2023-10-15 15:08:58,441][51532] Fps is (10 sec: 13110.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 17727488. Throughput: 0: 1766.2, 1: 1773.7. Samples: 4439304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:08:58,442][51532] Avg episode reward: [(0, '15.690'), (1, '16.450')] -[2023-10-15 15:09:00,469][52866] Updated weights for policy 1, policy_version 8680 (0.0007) -[2023-10-15 15:09:00,849][52866] Updated weights for policy 1, policy_version 8690 (0.0009) -[2023-10-15 15:09:01,209][52866] Updated weights for policy 1, policy_version 8700 (0.0010) -[2023-10-15 15:09:01,522][52833] Updated weights for policy 0, policy_version 8650 (0.0009) -[2023-10-15 15:09:01,894][52833] Updated weights for policy 0, policy_version 8660 (0.0008) -[2023-10-15 15:09:02,271][52833] Updated weights for policy 0, policy_version 8670 (0.0009) -[2023-10-15 15:09:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17793024. Throughput: 0: 1788.9, 1: 1784.1. Samples: 4450908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:09:03,442][51532] Avg episode reward: [(0, '14.400'), (1, '15.630')] -[2023-10-15 15:09:05,019][52866] Updated weights for policy 1, policy_version 8710 (0.0007) -[2023-10-15 15:09:05,380][52866] Updated weights for policy 1, policy_version 8720 (0.0008) -[2023-10-15 15:09:05,745][52866] Updated weights for policy 1, policy_version 8730 (0.0008) -[2023-10-15 15:09:06,130][52833] Updated weights for policy 0, policy_version 8680 (0.0008) -[2023-10-15 15:09:06,495][52833] Updated weights for policy 0, policy_version 8690 (0.0009) -[2023-10-15 15:09:06,857][52833] Updated weights for policy 0, policy_version 8700 (0.0010) -[2023-10-15 15:09:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17858560. Throughput: 0: 1776.5, 1: 1767.0. Samples: 4471270. Policy #0 lag: (min: 15.0, avg: 16.6, max: 42.0) -[2023-10-15 15:09:08,442][51532] Avg episode reward: [(0, '14.790'), (1, '16.040')] -[2023-10-15 15:09:09,421][52866] Updated weights for policy 1, policy_version 8740 (0.0007) -[2023-10-15 15:09:09,795][52866] Updated weights for policy 1, policy_version 8750 (0.0009) -[2023-10-15 15:09:10,155][52866] Updated weights for policy 1, policy_version 8760 (0.0008) -[2023-10-15 15:09:10,657][52833] Updated weights for policy 0, policy_version 8710 (0.0008) -[2023-10-15 15:09:11,044][52833] Updated weights for policy 0, policy_version 8720 (0.0008) -[2023-10-15 15:09:11,413][52833] Updated weights for policy 0, policy_version 8730 (0.0008) -[2023-10-15 15:09:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17924096. Throughput: 0: 1773.6, 1: 1780.5. Samples: 4493568. Policy #0 lag: (min: 15.0, avg: 16.6, max: 42.0) -[2023-10-15 15:09:13,441][51532] Avg episode reward: [(0, '17.380'), (1, '14.930')] -[2023-10-15 15:09:13,450][52410] Saving new best policy, reward=17.380! -[2023-10-15 15:09:14,014][52866] Updated weights for policy 1, policy_version 8770 (0.0007) -[2023-10-15 15:09:14,373][52866] Updated weights for policy 1, policy_version 8780 (0.0008) -[2023-10-15 15:09:14,738][52866] Updated weights for policy 1, policy_version 8790 (0.0007) -[2023-10-15 15:09:15,078][52833] Updated weights for policy 0, policy_version 8740 (0.0008) -[2023-10-15 15:09:15,106][52866] Updated weights for policy 1, policy_version 8800 (0.0008) -[2023-10-15 15:09:15,450][52833] Updated weights for policy 0, policy_version 8750 (0.0012) -[2023-10-15 15:09:15,832][52833] Updated weights for policy 0, policy_version 8760 (0.0010) -[2023-10-15 15:09:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 17989632. Throughput: 0: 1784.6, 1: 1771.3. Samples: 4503624. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) -[2023-10-15 15:09:18,441][51532] Avg episode reward: [(0, '15.490'), (1, '15.040')] -[2023-10-15 15:09:18,846][52866] Updated weights for policy 1, policy_version 8810 (0.0009) -[2023-10-15 15:09:19,213][52866] Updated weights for policy 1, policy_version 8820 (0.0008) -[2023-10-15 15:09:19,558][52833] Updated weights for policy 0, policy_version 8770 (0.0008) -[2023-10-15 15:09:19,577][52866] Updated weights for policy 1, policy_version 8830 (0.0008) -[2023-10-15 15:09:19,940][52833] Updated weights for policy 0, policy_version 8780 (0.0008) -[2023-10-15 15:09:20,298][52833] Updated weights for policy 0, policy_version 8790 (0.0011) -[2023-10-15 15:09:20,667][52833] Updated weights for policy 0, policy_version 8800 (0.0007) -[2023-10-15 15:09:23,318][52866] Updated weights for policy 1, policy_version 8840 (0.0009) -[2023-10-15 15:09:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 18055168. Throughput: 0: 1774.1, 1: 1770.5. Samples: 4525336. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) -[2023-10-15 15:09:23,441][51532] Avg episode reward: [(0, '17.400'), (1, '16.630')] -[2023-10-15 15:09:23,442][52410] Saving new best policy, reward=17.400! -[2023-10-15 15:09:23,683][52866] Updated weights for policy 1, policy_version 8850 (0.0011) -[2023-10-15 15:09:24,048][52866] Updated weights for policy 1, policy_version 8860 (0.0008) -[2023-10-15 15:09:24,542][52833] Updated weights for policy 0, policy_version 8810 (0.0010) -[2023-10-15 15:09:24,911][52833] Updated weights for policy 0, policy_version 8820 (0.0007) -[2023-10-15 15:09:25,284][52833] Updated weights for policy 0, policy_version 8830 (0.0008) -[2023-10-15 15:09:27,725][52866] Updated weights for policy 1, policy_version 8870 (0.0009) -[2023-10-15 15:09:28,103][52866] Updated weights for policy 1, policy_version 8880 (0.0010) -[2023-10-15 15:09:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 18120704. Throughput: 0: 1778.4, 1: 1789.1. Samples: 4547070. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 15:09:28,441][51532] Avg episode reward: [(0, '16.390'), (1, '16.870')] -[2023-10-15 15:09:28,480][52866] Updated weights for policy 1, policy_version 8890 (0.0009) -[2023-10-15 15:09:28,928][52833] Updated weights for policy 0, policy_version 8840 (0.0009) -[2023-10-15 15:09:29,297][52833] Updated weights for policy 0, policy_version 8850 (0.0009) -[2023-10-15 15:09:29,662][52833] Updated weights for policy 0, policy_version 8860 (0.0009) -[2023-10-15 15:09:32,393][52866] Updated weights for policy 1, policy_version 8900 (0.0008) -[2023-10-15 15:09:32,763][52866] Updated weights for policy 1, policy_version 8910 (0.0008) -[2023-10-15 15:09:33,144][52866] Updated weights for policy 1, policy_version 8920 (0.0008) -[2023-10-15 15:09:33,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 18219008. Throughput: 0: 1772.6, 1: 1769.8. Samples: 4557386. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 15:09:33,442][51532] Avg episode reward: [(0, '15.540'), (1, '16.920')] -[2023-10-15 15:09:33,452][52833] Updated weights for policy 0, policy_version 8870 (0.0007) -[2023-10-15 15:09:33,821][52833] Updated weights for policy 0, policy_version 8880 (0.0009) -[2023-10-15 15:09:34,185][52833] Updated weights for policy 0, policy_version 8890 (0.0011) -[2023-10-15 15:09:36,920][52866] Updated weights for policy 1, policy_version 8930 (0.0009) -[2023-10-15 15:09:37,291][52866] Updated weights for policy 1, policy_version 8940 (0.0009) -[2023-10-15 15:09:37,654][52866] Updated weights for policy 1, policy_version 8950 (0.0007) -[2023-10-15 15:09:38,023][52866] Updated weights for policy 1, policy_version 8960 (0.0008) -[2023-10-15 15:09:38,030][52833] Updated weights for policy 0, policy_version 8900 (0.0010) -[2023-10-15 15:09:38,408][52833] Updated weights for policy 0, policy_version 8910 (0.0008) -[2023-10-15 15:09:38,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 18284544. Throughput: 0: 1779.1, 1: 1793.1. Samples: 4579358. Policy #0 lag: (min: 17.0, avg: 17.6, max: 34.0) -[2023-10-15 15:09:38,442][51532] Avg episode reward: [(0, '16.390'), (1, '18.000')] -[2023-10-15 15:09:38,443][52518] Saving new best policy, reward=18.000! -[2023-10-15 15:09:38,775][52833] Updated weights for policy 0, policy_version 8920 (0.0008) -[2023-10-15 15:09:41,659][52866] Updated weights for policy 1, policy_version 8970 (0.0008) -[2023-10-15 15:09:42,031][52866] Updated weights for policy 1, policy_version 8980 (0.0007) -[2023-10-15 15:09:42,397][52866] Updated weights for policy 1, policy_version 8990 (0.0008) -[2023-10-15 15:09:42,482][52833] Updated weights for policy 0, policy_version 8930 (0.0009) -[2023-10-15 15:09:42,860][52833] Updated weights for policy 0, policy_version 8940 (0.0007) -[2023-10-15 15:09:43,233][52833] Updated weights for policy 0, policy_version 8950 (0.0008) -[2023-10-15 15:09:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.3, 300 sec: 14106.9). Total num frames: 18350080. Throughput: 0: 1796.7, 1: 1775.4. Samples: 4600050. Policy #0 lag: (min: 17.0, avg: 17.6, max: 34.0) -[2023-10-15 15:09:43,443][51532] Avg episode reward: [(0, '15.150'), (1, '15.820')] -[2023-10-15 15:09:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000008992_9207808.pth... -[2023-10-15 15:09:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000007328_7503872.pth -[2023-10-15 15:09:43,605][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000008960_9175040.pth... -[2023-10-15 15:09:43,608][52833] Updated weights for policy 0, policy_version 8960 (0.0010) -[2023-10-15 15:09:43,634][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000007296_7471104.pth -[2023-10-15 15:09:46,205][52866] Updated weights for policy 1, policy_version 9000 (0.0008) -[2023-10-15 15:09:46,578][52866] Updated weights for policy 1, policy_version 9010 (0.0009) -[2023-10-15 15:09:46,949][52866] Updated weights for policy 1, policy_version 9020 (0.0008) -[2023-10-15 15:09:47,385][52833] Updated weights for policy 0, policy_version 8970 (0.0008) -[2023-10-15 15:09:47,762][52833] Updated weights for policy 0, policy_version 8980 (0.0007) -[2023-10-15 15:09:48,136][52833] Updated weights for policy 0, policy_version 8990 (0.0011) -[2023-10-15 15:09:48,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.9, 300 sec: 14218.0). Total num frames: 18448384. Throughput: 0: 1772.4, 1: 1795.9. Samples: 4611482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:09:48,442][51532] Avg episode reward: [(0, '15.760'), (1, '15.520')] -[2023-10-15 15:09:50,730][52866] Updated weights for policy 1, policy_version 9030 (0.0008) -[2023-10-15 15:09:51,102][52866] Updated weights for policy 1, policy_version 9040 (0.0008) -[2023-10-15 15:09:51,468][52866] Updated weights for policy 1, policy_version 9050 (0.0008) -[2023-10-15 15:09:51,954][52833] Updated weights for policy 0, policy_version 9000 (0.0010) -[2023-10-15 15:09:52,329][52833] Updated weights for policy 0, policy_version 9010 (0.0008) -[2023-10-15 15:09:52,690][52833] Updated weights for policy 0, policy_version 9020 (0.0008) -[2023-10-15 15:09:53,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 18513920. Throughput: 0: 1794.2, 1: 1779.5. Samples: 4632084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:09:53,442][51532] Avg episode reward: [(0, '16.170'), (1, '16.200')] -[2023-10-15 15:09:55,254][52866] Updated weights for policy 1, policy_version 9060 (0.0008) -[2023-10-15 15:09:55,611][52866] Updated weights for policy 1, policy_version 9070 (0.0007) -[2023-10-15 15:09:55,974][52866] Updated weights for policy 1, policy_version 9080 (0.0007) -[2023-10-15 15:09:56,571][52833] Updated weights for policy 0, policy_version 9030 (0.0011) -[2023-10-15 15:09:56,952][52833] Updated weights for policy 0, policy_version 9040 (0.0011) -[2023-10-15 15:09:57,330][52833] Updated weights for policy 0, policy_version 9050 (0.0008) -[2023-10-15 15:09:58,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 18579456. Throughput: 0: 1765.7, 1: 1774.2. Samples: 4652864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:09:58,442][51532] Avg episode reward: [(0, '16.570'), (1, '15.840')] -[2023-10-15 15:09:59,892][52866] Updated weights for policy 1, policy_version 9090 (0.0007) -[2023-10-15 15:10:00,251][52866] Updated weights for policy 1, policy_version 9100 (0.0007) -[2023-10-15 15:10:00,626][52866] Updated weights for policy 1, policy_version 9110 (0.0009) -[2023-10-15 15:10:00,992][52866] Updated weights for policy 1, policy_version 9120 (0.0011) -[2023-10-15 15:10:01,306][52833] Updated weights for policy 0, policy_version 9060 (0.0009) -[2023-10-15 15:10:01,673][52833] Updated weights for policy 0, policy_version 9070 (0.0008) -[2023-10-15 15:10:02,053][52833] Updated weights for policy 0, policy_version 9080 (0.0007) -[2023-10-15 15:10:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 18644992. Throughput: 0: 1785.6, 1: 1780.7. Samples: 4664108. Policy #0 lag: (min: 26.0, avg: 34.6, max: 58.0) -[2023-10-15 15:10:03,442][51532] Avg episode reward: [(0, '16.630'), (1, '16.730')] -[2023-10-15 15:10:04,804][52866] Updated weights for policy 1, policy_version 9130 (0.0008) -[2023-10-15 15:10:05,173][52866] Updated weights for policy 1, policy_version 9140 (0.0009) -[2023-10-15 15:10:05,537][52866] Updated weights for policy 1, policy_version 9150 (0.0008) -[2023-10-15 15:10:05,751][52833] Updated weights for policy 0, policy_version 9090 (0.0007) -[2023-10-15 15:10:06,117][52833] Updated weights for policy 0, policy_version 9100 (0.0008) -[2023-10-15 15:10:06,494][52833] Updated weights for policy 0, policy_version 9110 (0.0008) -[2023-10-15 15:10:06,859][52833] Updated weights for policy 0, policy_version 9120 (0.0007) -[2023-10-15 15:10:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 18710528. Throughput: 0: 1769.6, 1: 1775.9. Samples: 4684884. Policy #0 lag: (min: 26.0, avg: 34.6, max: 58.0) -[2023-10-15 15:10:08,442][51532] Avg episode reward: [(0, '15.380'), (1, '16.540')] -[2023-10-15 15:10:09,395][52866] Updated weights for policy 1, policy_version 9160 (0.0007) -[2023-10-15 15:10:09,768][52866] Updated weights for policy 1, policy_version 9170 (0.0009) -[2023-10-15 15:10:10,140][52866] Updated weights for policy 1, policy_version 9180 (0.0007) -[2023-10-15 15:10:10,643][52833] Updated weights for policy 0, policy_version 9130 (0.0008) -[2023-10-15 15:10:11,027][52833] Updated weights for policy 0, policy_version 9140 (0.0009) -[2023-10-15 15:10:11,384][52833] Updated weights for policy 0, policy_version 9150 (0.0009) -[2023-10-15 15:10:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 18776064. Throughput: 0: 1766.0, 1: 1793.2. Samples: 4707236. Policy #0 lag: (min: 31.0, avg: 32.3, max: 54.0) -[2023-10-15 15:10:13,442][51532] Avg episode reward: [(0, '16.220'), (1, '16.200')] -[2023-10-15 15:10:13,757][52866] Updated weights for policy 1, policy_version 9190 (0.0008) -[2023-10-15 15:10:14,128][52866] Updated weights for policy 1, policy_version 9200 (0.0010) -[2023-10-15 15:10:14,491][52866] Updated weights for policy 1, policy_version 9210 (0.0010) -[2023-10-15 15:10:15,245][52833] Updated weights for policy 0, policy_version 9160 (0.0010) -[2023-10-15 15:10:15,617][52833] Updated weights for policy 0, policy_version 9170 (0.0009) -[2023-10-15 15:10:15,991][52833] Updated weights for policy 0, policy_version 9180 (0.0011) -[2023-10-15 15:10:18,312][52866] Updated weights for policy 1, policy_version 9220 (0.0010) -[2023-10-15 15:10:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 18841600. Throughput: 0: 1769.6, 1: 1783.9. Samples: 4717292. Policy #0 lag: (min: 31.0, avg: 32.3, max: 54.0) -[2023-10-15 15:10:18,441][51532] Avg episode reward: [(0, '16.820'), (1, '16.060')] -[2023-10-15 15:10:18,683][52866] Updated weights for policy 1, policy_version 9230 (0.0008) -[2023-10-15 15:10:19,048][52866] Updated weights for policy 1, policy_version 9240 (0.0007) -[2023-10-15 15:10:19,812][52833] Updated weights for policy 0, policy_version 9190 (0.0008) -[2023-10-15 15:10:20,182][52833] Updated weights for policy 0, policy_version 9200 (0.0007) -[2023-10-15 15:10:20,557][52833] Updated weights for policy 0, policy_version 9210 (0.0008) -[2023-10-15 15:10:22,811][52866] Updated weights for policy 1, policy_version 9250 (0.0007) -[2023-10-15 15:10:23,193][52866] Updated weights for policy 1, policy_version 9260 (0.0010) -[2023-10-15 15:10:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 18907136. Throughput: 0: 1758.1, 1: 1790.1. Samples: 4739030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:10:23,442][51532] Avg episode reward: [(0, '15.230'), (1, '15.290')] -[2023-10-15 15:10:23,559][52866] Updated weights for policy 1, policy_version 9270 (0.0007) -[2023-10-15 15:10:23,936][52866] Updated weights for policy 1, policy_version 9280 (0.0009) -[2023-10-15 15:10:24,214][52833] Updated weights for policy 0, policy_version 9220 (0.0009) -[2023-10-15 15:10:24,580][52833] Updated weights for policy 0, policy_version 9230 (0.0010) -[2023-10-15 15:10:24,940][52833] Updated weights for policy 0, policy_version 9240 (0.0010) -[2023-10-15 15:10:27,673][52866] Updated weights for policy 1, policy_version 9290 (0.0011) -[2023-10-15 15:10:28,043][52866] Updated weights for policy 1, policy_version 9300 (0.0009) -[2023-10-15 15:10:28,415][52866] Updated weights for policy 1, policy_version 9310 (0.0009) -[2023-10-15 15:10:28,441][51532] Fps is (10 sec: 13106.3, 60 sec: 14199.3, 300 sec: 14218.0). Total num frames: 18972672. Throughput: 0: 1781.0, 1: 1794.8. Samples: 4760960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:10:28,443][51532] Avg episode reward: [(0, '17.490'), (1, '16.340')] -[2023-10-15 15:10:28,453][52410] Saving new best policy, reward=17.490! -[2023-10-15 15:10:28,691][52833] Updated weights for policy 0, policy_version 9250 (0.0010) -[2023-10-15 15:10:29,062][52833] Updated weights for policy 0, policy_version 9260 (0.0009) -[2023-10-15 15:10:29,437][52833] Updated weights for policy 0, policy_version 9270 (0.0009) -[2023-10-15 15:10:29,804][52833] Updated weights for policy 0, policy_version 9280 (0.0007) -[2023-10-15 15:10:32,291][52866] Updated weights for policy 1, policy_version 9320 (0.0007) -[2023-10-15 15:10:32,655][52866] Updated weights for policy 1, policy_version 9330 (0.0008) -[2023-10-15 15:10:33,018][52866] Updated weights for policy 1, policy_version 9340 (0.0007) -[2023-10-15 15:10:33,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 19070976. Throughput: 0: 1774.9, 1: 1782.5. Samples: 4771562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:10:33,442][51532] Avg episode reward: [(0, '16.660'), (1, '15.760')] -[2023-10-15 15:10:33,532][52833] Updated weights for policy 0, policy_version 9290 (0.0009) -[2023-10-15 15:10:33,903][52833] Updated weights for policy 0, policy_version 9300 (0.0011) -[2023-10-15 15:10:34,275][52833] Updated weights for policy 0, policy_version 9310 (0.0010) -[2023-10-15 15:10:36,488][52866] Updated weights for policy 1, policy_version 9350 (0.0009) -[2023-10-15 15:10:36,859][52866] Updated weights for policy 1, policy_version 9360 (0.0010) -[2023-10-15 15:10:37,220][52866] Updated weights for policy 1, policy_version 9370 (0.0010) -[2023-10-15 15:10:38,015][52833] Updated weights for policy 0, policy_version 9320 (0.0008) -[2023-10-15 15:10:38,389][52833] Updated weights for policy 0, policy_version 9330 (0.0009) -[2023-10-15 15:10:38,441][51532] Fps is (10 sec: 16384.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 19136512. Throughput: 0: 1776.5, 1: 1798.9. Samples: 4792972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:10:38,442][51532] Avg episode reward: [(0, '16.160'), (1, '16.720')] -[2023-10-15 15:10:38,748][52833] Updated weights for policy 0, policy_version 9340 (0.0010) -[2023-10-15 15:10:41,061][52866] Updated weights for policy 1, policy_version 9380 (0.0009) -[2023-10-15 15:10:41,424][52866] Updated weights for policy 1, policy_version 9390 (0.0010) -[2023-10-15 15:10:41,794][52866] Updated weights for policy 1, policy_version 9400 (0.0009) -[2023-10-15 15:10:42,607][52833] Updated weights for policy 0, policy_version 9350 (0.0009) -[2023-10-15 15:10:42,998][52833] Updated weights for policy 0, policy_version 9360 (0.0008) -[2023-10-15 15:10:43,361][52833] Updated weights for policy 0, policy_version 9370 (0.0008) -[2023-10-15 15:10:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14218.0). Total num frames: 19202048. Throughput: 0: 1797.2, 1: 1789.2. Samples: 4814250. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) -[2023-10-15 15:10:43,441][51532] Avg episode reward: [(0, '16.930'), (1, '16.600')] -[2023-10-15 15:10:45,573][52866] Updated weights for policy 1, policy_version 9410 (0.0009) -[2023-10-15 15:10:45,936][52866] Updated weights for policy 1, policy_version 9420 (0.0009) -[2023-10-15 15:10:46,291][52866] Updated weights for policy 1, policy_version 9430 (0.0008) -[2023-10-15 15:10:46,657][52866] Updated weights for policy 1, policy_version 9440 (0.0007) -[2023-10-15 15:10:47,097][52833] Updated weights for policy 0, policy_version 9380 (0.0009) -[2023-10-15 15:10:47,467][52833] Updated weights for policy 0, policy_version 9390 (0.0008) -[2023-10-15 15:10:47,843][52833] Updated weights for policy 0, policy_version 9400 (0.0007) -[2023-10-15 15:10:48,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.6, 300 sec: 14218.0). Total num frames: 19300352. Throughput: 0: 1776.0, 1: 1806.1. Samples: 4825304. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) -[2023-10-15 15:10:48,442][51532] Avg episode reward: [(0, '17.330'), (1, '15.180')] -[2023-10-15 15:10:50,408][52866] Updated weights for policy 1, policy_version 9450 (0.0008) -[2023-10-15 15:10:50,776][52866] Updated weights for policy 1, policy_version 9460 (0.0008) -[2023-10-15 15:10:51,142][52866] Updated weights for policy 1, policy_version 9470 (0.0008) -[2023-10-15 15:10:51,648][52833] Updated weights for policy 0, policy_version 9410 (0.0008) -[2023-10-15 15:10:52,018][52833] Updated weights for policy 0, policy_version 9420 (0.0007) -[2023-10-15 15:10:52,398][52833] Updated weights for policy 0, policy_version 9430 (0.0007) -[2023-10-15 15:10:52,762][52833] Updated weights for policy 0, policy_version 9440 (0.0007) -[2023-10-15 15:10:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 19365888. Throughput: 0: 1798.5, 1: 1792.5. Samples: 4846474. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) -[2023-10-15 15:10:53,441][51532] Avg episode reward: [(0, '16.860'), (1, '15.320')] -[2023-10-15 15:10:54,939][52866] Updated weights for policy 1, policy_version 9480 (0.0010) -[2023-10-15 15:10:55,309][52866] Updated weights for policy 1, policy_version 9490 (0.0009) -[2023-10-15 15:10:55,673][52866] Updated weights for policy 1, policy_version 9500 (0.0010) -[2023-10-15 15:10:56,683][52833] Updated weights for policy 0, policy_version 9450 (0.0009) -[2023-10-15 15:10:57,052][52833] Updated weights for policy 0, policy_version 9460 (0.0009) -[2023-10-15 15:10:57,412][52833] Updated weights for policy 0, policy_version 9470 (0.0008) -[2023-10-15 15:10:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 19431424. Throughput: 0: 1773.3, 1: 1788.8. Samples: 4867530. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 15:10:58,442][51532] Avg episode reward: [(0, '16.130'), (1, '15.880')] -[2023-10-15 15:10:59,442][52866] Updated weights for policy 1, policy_version 9510 (0.0009) -[2023-10-15 15:10:59,816][52866] Updated weights for policy 1, policy_version 9520 (0.0007) -[2023-10-15 15:11:00,186][52866] Updated weights for policy 1, policy_version 9530 (0.0007) -[2023-10-15 15:11:01,174][52833] Updated weights for policy 0, policy_version 9480 (0.0010) -[2023-10-15 15:11:01,548][52833] Updated weights for policy 0, policy_version 9490 (0.0009) -[2023-10-15 15:11:01,922][52833] Updated weights for policy 0, policy_version 9500 (0.0007) -[2023-10-15 15:11:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 19496960. Throughput: 0: 1803.9, 1: 1787.0. Samples: 4878882. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 15:11:03,442][51532] Avg episode reward: [(0, '16.300'), (1, '15.990')] -[2023-10-15 15:11:03,915][52866] Updated weights for policy 1, policy_version 9540 (0.0009) -[2023-10-15 15:11:04,284][52866] Updated weights for policy 1, policy_version 9550 (0.0010) -[2023-10-15 15:11:04,649][52866] Updated weights for policy 1, policy_version 9560 (0.0010) -[2023-10-15 15:11:05,504][52833] Updated weights for policy 0, policy_version 9510 (0.0008) -[2023-10-15 15:11:05,878][52833] Updated weights for policy 0, policy_version 9520 (0.0009) -[2023-10-15 15:11:06,242][52833] Updated weights for policy 0, policy_version 9530 (0.0008) -[2023-10-15 15:11:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 19562496. Throughput: 0: 1785.6, 1: 1787.9. Samples: 4899838. Policy #0 lag: (min: 9.0, avg: 25.3, max: 41.0) -[2023-10-15 15:11:08,442][51532] Avg episode reward: [(0, '15.870'), (1, '16.050')] -[2023-10-15 15:11:08,465][52866] Updated weights for policy 1, policy_version 9570 (0.0009) -[2023-10-15 15:11:08,824][52866] Updated weights for policy 1, policy_version 9580 (0.0008) -[2023-10-15 15:11:09,190][52866] Updated weights for policy 1, policy_version 9590 (0.0008) -[2023-10-15 15:11:09,560][52866] Updated weights for policy 1, policy_version 9600 (0.0009) -[2023-10-15 15:11:10,150][52833] Updated weights for policy 0, policy_version 9540 (0.0008) -[2023-10-15 15:11:10,514][52833] Updated weights for policy 0, policy_version 9550 (0.0008) -[2023-10-15 15:11:10,885][52833] Updated weights for policy 0, policy_version 9560 (0.0007) -[2023-10-15 15:11:13,217][52866] Updated weights for policy 1, policy_version 9610 (0.0009) -[2023-10-15 15:11:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 19628032. Throughput: 0: 1770.2, 1: 1809.1. Samples: 4922028. Policy #0 lag: (min: 9.0, avg: 25.3, max: 41.0) -[2023-10-15 15:11:13,441][51532] Avg episode reward: [(0, '16.670'), (1, '16.710')] -[2023-10-15 15:11:13,581][52866] Updated weights for policy 1, policy_version 9620 (0.0008) -[2023-10-15 15:11:13,948][52866] Updated weights for policy 1, policy_version 9630 (0.0009) -[2023-10-15 15:11:14,683][52833] Updated weights for policy 0, policy_version 9570 (0.0010) -[2023-10-15 15:11:15,054][52833] Updated weights for policy 0, policy_version 9580 (0.0009) -[2023-10-15 15:11:15,422][52833] Updated weights for policy 0, policy_version 9590 (0.0009) -[2023-10-15 15:11:15,790][52833] Updated weights for policy 0, policy_version 9600 (0.0008) -[2023-10-15 15:11:17,752][52866] Updated weights for policy 1, policy_version 9640 (0.0011) -[2023-10-15 15:11:18,129][52866] Updated weights for policy 1, policy_version 9650 (0.0011) -[2023-10-15 15:11:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 19693568. Throughput: 0: 1770.3, 1: 1795.1. Samples: 4932002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:11:18,441][51532] Avg episode reward: [(0, '16.450'), (1, '15.990')] -[2023-10-15 15:11:18,492][52866] Updated weights for policy 1, policy_version 9660 (0.0010) -[2023-10-15 15:11:19,523][52833] Updated weights for policy 0, policy_version 9610 (0.0009) -[2023-10-15 15:11:19,903][52833] Updated weights for policy 0, policy_version 9620 (0.0011) -[2023-10-15 15:11:20,262][52833] Updated weights for policy 0, policy_version 9630 (0.0010) -[2023-10-15 15:11:22,277][52866] Updated weights for policy 1, policy_version 9670 (0.0010) -[2023-10-15 15:11:22,644][52866] Updated weights for policy 1, policy_version 9680 (0.0010) -[2023-10-15 15:11:23,002][52866] Updated weights for policy 1, policy_version 9690 (0.0008) -[2023-10-15 15:11:23,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 19791872. Throughput: 0: 1772.6, 1: 1808.2. Samples: 4954108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:11:23,442][51532] Avg episode reward: [(0, '16.450'), (1, '16.500')] -[2023-10-15 15:11:24,046][52833] Updated weights for policy 0, policy_version 9640 (0.0009) -[2023-10-15 15:11:24,418][52833] Updated weights for policy 0, policy_version 9650 (0.0008) -[2023-10-15 15:11:24,790][52833] Updated weights for policy 0, policy_version 9660 (0.0010) -[2023-10-15 15:11:26,743][52866] Updated weights for policy 1, policy_version 9700 (0.0010) -[2023-10-15 15:11:27,111][52866] Updated weights for policy 1, policy_version 9710 (0.0009) -[2023-10-15 15:11:27,487][52866] Updated weights for policy 1, policy_version 9720 (0.0008) -[2023-10-15 15:11:28,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.7, 300 sec: 14329.0). Total num frames: 19857408. Throughput: 0: 1787.3, 1: 1787.2. Samples: 4975104. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) -[2023-10-15 15:11:28,442][51532] Avg episode reward: [(0, '17.060'), (1, '17.400')] -[2023-10-15 15:11:28,518][52833] Updated weights for policy 0, policy_version 9670 (0.0008) -[2023-10-15 15:11:28,898][52833] Updated weights for policy 0, policy_version 9680 (0.0007) -[2023-10-15 15:11:29,261][52833] Updated weights for policy 0, policy_version 9690 (0.0008) -[2023-10-15 15:11:31,229][52866] Updated weights for policy 1, policy_version 9730 (0.0008) -[2023-10-15 15:11:31,590][52866] Updated weights for policy 1, policy_version 9740 (0.0011) -[2023-10-15 15:11:31,964][52866] Updated weights for policy 1, policy_version 9750 (0.0010) -[2023-10-15 15:11:32,325][52866] Updated weights for policy 1, policy_version 9760 (0.0011) -[2023-10-15 15:11:33,095][52833] Updated weights for policy 0, policy_version 9700 (0.0008) -[2023-10-15 15:11:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 19922944. Throughput: 0: 1773.2, 1: 1803.2. Samples: 4986242. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) -[2023-10-15 15:11:33,442][51532] Avg episode reward: [(0, '18.620'), (1, '16.930')] -[2023-10-15 15:11:33,454][52833] Updated weights for policy 0, policy_version 9710 (0.0010) -[2023-10-15 15:11:33,817][52833] Updated weights for policy 0, policy_version 9720 (0.0010) -[2023-10-15 15:11:34,114][52410] Saving new best policy, reward=18.620! -[2023-10-15 15:11:35,901][52866] Updated weights for policy 1, policy_version 9770 (0.0008) -[2023-10-15 15:11:36,273][52866] Updated weights for policy 1, policy_version 9780 (0.0008) -[2023-10-15 15:11:36,643][52866] Updated weights for policy 1, policy_version 9790 (0.0010) -[2023-10-15 15:11:37,678][52833] Updated weights for policy 0, policy_version 9730 (0.0010) -[2023-10-15 15:11:38,046][52833] Updated weights for policy 0, policy_version 9740 (0.0007) -[2023-10-15 15:11:38,413][52833] Updated weights for policy 0, policy_version 9750 (0.0008) -[2023-10-15 15:11:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 19988480. Throughput: 0: 1782.1, 1: 1794.8. Samples: 5007436. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) -[2023-10-15 15:11:38,442][51532] Avg episode reward: [(0, '17.450'), (1, '16.560')] -[2023-10-15 15:11:38,785][52833] Updated weights for policy 0, policy_version 9760 (0.0009) -[2023-10-15 15:11:40,356][52866] Updated weights for policy 1, policy_version 9800 (0.0007) -[2023-10-15 15:11:40,726][52866] Updated weights for policy 1, policy_version 9810 (0.0010) -[2023-10-15 15:11:41,090][52866] Updated weights for policy 1, policy_version 9820 (0.0011) -[2023-10-15 15:11:42,553][52833] Updated weights for policy 0, policy_version 9770 (0.0007) -[2023-10-15 15:11:42,922][52833] Updated weights for policy 0, policy_version 9780 (0.0007) -[2023-10-15 15:11:43,287][52833] Updated weights for policy 0, policy_version 9790 (0.0007) -[2023-10-15 15:11:43,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 20086784. Throughput: 0: 1792.7, 1: 1797.9. Samples: 5029106. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) -[2023-10-15 15:11:43,441][51532] Avg episode reward: [(0, '18.050'), (1, '17.240')] -[2023-10-15 15:11:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000009824_10059776.pth... -[2023-10-15 15:11:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000009792_10027008.pth... -[2023-10-15 15:11:43,487][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000008128_8323072.pth -[2023-10-15 15:11:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000008160_8355840.pth -[2023-10-15 15:11:44,819][52866] Updated weights for policy 1, policy_version 9830 (0.0009) -[2023-10-15 15:11:45,182][52866] Updated weights for policy 1, policy_version 9840 (0.0007) -[2023-10-15 15:11:45,555][52866] Updated weights for policy 1, policy_version 9850 (0.0008) -[2023-10-15 15:11:47,055][52833] Updated weights for policy 0, policy_version 9800 (0.0008) -[2023-10-15 15:11:47,427][52833] Updated weights for policy 0, policy_version 9810 (0.0008) -[2023-10-15 15:11:47,789][52833] Updated weights for policy 0, policy_version 9820 (0.0010) -[2023-10-15 15:11:48,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 20152320. Throughput: 0: 1778.3, 1: 1797.1. Samples: 5039774. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) -[2023-10-15 15:11:48,442][51532] Avg episode reward: [(0, '18.010'), (1, '16.440')] -[2023-10-15 15:11:49,324][52866] Updated weights for policy 1, policy_version 9860 (0.0008) -[2023-10-15 15:11:49,684][52866] Updated weights for policy 1, policy_version 9870 (0.0008) -[2023-10-15 15:11:50,060][52866] Updated weights for policy 1, policy_version 9880 (0.0008) -[2023-10-15 15:11:51,454][52833] Updated weights for policy 0, policy_version 9830 (0.0009) -[2023-10-15 15:11:51,816][52833] Updated weights for policy 0, policy_version 9840 (0.0007) -[2023-10-15 15:11:52,195][52833] Updated weights for policy 0, policy_version 9850 (0.0008) -[2023-10-15 15:11:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 20217856. Throughput: 0: 1793.3, 1: 1796.5. Samples: 5061374. Policy #0 lag: (min: 3.0, avg: 8.8, max: 35.0) -[2023-10-15 15:11:53,441][51532] Avg episode reward: [(0, '17.890'), (1, '16.630')] -[2023-10-15 15:11:53,736][52866] Updated weights for policy 1, policy_version 9890 (0.0010) -[2023-10-15 15:11:54,109][52866] Updated weights for policy 1, policy_version 9900 (0.0010) -[2023-10-15 15:11:54,486][52866] Updated weights for policy 1, policy_version 9910 (0.0008) -[2023-10-15 15:11:54,848][52866] Updated weights for policy 1, policy_version 9920 (0.0009) -[2023-10-15 15:11:56,013][52833] Updated weights for policy 0, policy_version 9860 (0.0008) -[2023-10-15 15:11:56,389][52833] Updated weights for policy 0, policy_version 9870 (0.0010) -[2023-10-15 15:11:56,756][52833] Updated weights for policy 0, policy_version 9880 (0.0010) -[2023-10-15 15:11:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 20283392. Throughput: 0: 1779.9, 1: 1801.7. Samples: 5083200. Policy #0 lag: (min: 3.0, avg: 8.8, max: 35.0) -[2023-10-15 15:11:58,442][51532] Avg episode reward: [(0, '16.830'), (1, '16.810')] -[2023-10-15 15:11:58,636][52866] Updated weights for policy 1, policy_version 9930 (0.0008) -[2023-10-15 15:11:59,004][52866] Updated weights for policy 1, policy_version 9940 (0.0008) -[2023-10-15 15:11:59,371][52866] Updated weights for policy 1, policy_version 9950 (0.0009) -[2023-10-15 15:12:00,482][52833] Updated weights for policy 0, policy_version 9890 (0.0007) -[2023-10-15 15:12:00,853][52833] Updated weights for policy 0, policy_version 9900 (0.0009) -[2023-10-15 15:12:01,227][52833] Updated weights for policy 0, policy_version 9910 (0.0008) -[2023-10-15 15:12:01,591][52833] Updated weights for policy 0, policy_version 9920 (0.0011) -[2023-10-15 15:12:03,185][52866] Updated weights for policy 1, policy_version 9960 (0.0008) -[2023-10-15 15:12:03,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 20348928. Throughput: 0: 1799.4, 1: 1793.0. Samples: 5093660. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) -[2023-10-15 15:12:03,443][51532] Avg episode reward: [(0, '16.200'), (1, '16.000')] -[2023-10-15 15:12:03,548][52866] Updated weights for policy 1, policy_version 9970 (0.0010) -[2023-10-15 15:12:03,913][52866] Updated weights for policy 1, policy_version 9980 (0.0009) -[2023-10-15 15:12:05,350][52833] Updated weights for policy 0, policy_version 9930 (0.0008) -[2023-10-15 15:12:05,727][52833] Updated weights for policy 0, policy_version 9940 (0.0007) -[2023-10-15 15:12:06,098][52833] Updated weights for policy 0, policy_version 9950 (0.0008) -[2023-10-15 15:12:07,686][52866] Updated weights for policy 1, policy_version 9990 (0.0008) -[2023-10-15 15:12:08,055][52866] Updated weights for policy 1, policy_version 10000 (0.0008) -[2023-10-15 15:12:08,428][52866] Updated weights for policy 1, policy_version 10010 (0.0008) -[2023-10-15 15:12:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 20414464. Throughput: 0: 1781.4, 1: 1798.4. Samples: 5115202. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) -[2023-10-15 15:12:08,442][51532] Avg episode reward: [(0, '16.490'), (1, '16.540')] -[2023-10-15 15:12:09,889][52833] Updated weights for policy 0, policy_version 9960 (0.0010) -[2023-10-15 15:12:10,252][52833] Updated weights for policy 0, policy_version 9970 (0.0007) -[2023-10-15 15:12:10,630][52833] Updated weights for policy 0, policy_version 9980 (0.0008) -[2023-10-15 15:12:12,127][52866] Updated weights for policy 1, policy_version 10020 (0.0008) -[2023-10-15 15:12:12,497][52866] Updated weights for policy 1, policy_version 10030 (0.0007) -[2023-10-15 15:12:12,867][52866] Updated weights for policy 1, policy_version 10040 (0.0008) -[2023-10-15 15:12:13,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 20512768. Throughput: 0: 1785.2, 1: 1803.7. Samples: 5136604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) -[2023-10-15 15:12:13,442][51532] Avg episode reward: [(0, '16.750'), (1, '16.010')] -[2023-10-15 15:12:14,262][52833] Updated weights for policy 0, policy_version 9990 (0.0009) -[2023-10-15 15:12:14,626][52833] Updated weights for policy 0, policy_version 10000 (0.0008) -[2023-10-15 15:12:14,992][52833] Updated weights for policy 0, policy_version 10010 (0.0009) -[2023-10-15 15:12:16,754][52866] Updated weights for policy 1, policy_version 10050 (0.0010) -[2023-10-15 15:12:17,120][52866] Updated weights for policy 1, policy_version 10060 (0.0009) -[2023-10-15 15:12:17,484][52866] Updated weights for policy 1, policy_version 10070 (0.0007) -[2023-10-15 15:12:17,853][52866] Updated weights for policy 1, policy_version 10080 (0.0008) -[2023-10-15 15:12:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 20578304. Throughput: 0: 1790.1, 1: 1793.4. Samples: 5147500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) -[2023-10-15 15:12:18,442][51532] Avg episode reward: [(0, '16.700'), (1, '15.950')] -[2023-10-15 15:12:18,786][52833] Updated weights for policy 0, policy_version 10020 (0.0009) -[2023-10-15 15:12:19,152][52833] Updated weights for policy 0, policy_version 10030 (0.0010) -[2023-10-15 15:12:19,520][52833] Updated weights for policy 0, policy_version 10040 (0.0011) -[2023-10-15 15:12:21,756][52866] Updated weights for policy 1, policy_version 10090 (0.0012) -[2023-10-15 15:12:22,118][52866] Updated weights for policy 1, policy_version 10100 (0.0011) -[2023-10-15 15:12:22,492][52866] Updated weights for policy 1, policy_version 10110 (0.0009) -[2023-10-15 15:12:23,363][52833] Updated weights for policy 0, policy_version 10050 (0.0010) -[2023-10-15 15:12:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 20643840. Throughput: 0: 1781.6, 1: 1804.1. Samples: 5168792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:12:23,442][51532] Avg episode reward: [(0, '18.140'), (1, '16.710')] -[2023-10-15 15:12:23,725][52833] Updated weights for policy 0, policy_version 10060 (0.0007) -[2023-10-15 15:12:24,091][52833] Updated weights for policy 0, policy_version 10070 (0.0008) -[2023-10-15 15:12:24,457][52833] Updated weights for policy 0, policy_version 10080 (0.0008) -[2023-10-15 15:12:26,319][52866] Updated weights for policy 1, policy_version 10120 (0.0011) -[2023-10-15 15:12:26,680][52866] Updated weights for policy 1, policy_version 10130 (0.0010) -[2023-10-15 15:12:27,058][52866] Updated weights for policy 1, policy_version 10140 (0.0012) -[2023-10-15 15:12:28,341][52833] Updated weights for policy 0, policy_version 10090 (0.0011) -[2023-10-15 15:12:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 20709376. Throughput: 0: 1796.5, 1: 1772.8. Samples: 5189724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:12:28,441][51532] Avg episode reward: [(0, '17.100'), (1, '15.560')] -[2023-10-15 15:12:28,714][52833] Updated weights for policy 0, policy_version 10100 (0.0011) -[2023-10-15 15:12:29,076][52833] Updated weights for policy 0, policy_version 10110 (0.0011) -[2023-10-15 15:12:30,908][52866] Updated weights for policy 1, policy_version 10150 (0.0008) -[2023-10-15 15:12:31,278][52866] Updated weights for policy 1, policy_version 10160 (0.0009) -[2023-10-15 15:12:31,631][52866] Updated weights for policy 1, policy_version 10170 (0.0007) -[2023-10-15 15:12:33,053][52833] Updated weights for policy 0, policy_version 10120 (0.0008) -[2023-10-15 15:12:33,431][52833] Updated weights for policy 0, policy_version 10130 (0.0008) -[2023-10-15 15:12:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 20774912. Throughput: 0: 1773.8, 1: 1798.0. Samples: 5200504. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) -[2023-10-15 15:12:33,441][51532] Avg episode reward: [(0, '16.090'), (1, '15.200')] -[2023-10-15 15:12:33,805][52833] Updated weights for policy 0, policy_version 10140 (0.0011) -[2023-10-15 15:12:35,333][52866] Updated weights for policy 1, policy_version 10180 (0.0009) -[2023-10-15 15:12:35,714][52866] Updated weights for policy 1, policy_version 10190 (0.0010) -[2023-10-15 15:12:36,087][52866] Updated weights for policy 1, policy_version 10200 (0.0013) -[2023-10-15 15:12:37,506][52833] Updated weights for policy 0, policy_version 10150 (0.0009) -[2023-10-15 15:12:37,880][52833] Updated weights for policy 0, policy_version 10160 (0.0007) -[2023-10-15 15:12:38,244][52833] Updated weights for policy 0, policy_version 10170 (0.0007) -[2023-10-15 15:12:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 20840448. Throughput: 0: 1792.8, 1: 1773.0. Samples: 5221834. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) -[2023-10-15 15:12:38,442][51532] Avg episode reward: [(0, '14.830'), (1, '16.520')] -[2023-10-15 15:12:39,953][52866] Updated weights for policy 1, policy_version 10210 (0.0008) -[2023-10-15 15:12:40,327][52866] Updated weights for policy 1, policy_version 10220 (0.0007) -[2023-10-15 15:12:40,691][52866] Updated weights for policy 1, policy_version 10230 (0.0010) -[2023-10-15 15:12:41,055][52866] Updated weights for policy 1, policy_version 10240 (0.0011) -[2023-10-15 15:12:41,899][52833] Updated weights for policy 0, policy_version 10180 (0.0008) -[2023-10-15 15:12:42,270][52833] Updated weights for policy 0, policy_version 10190 (0.0009) -[2023-10-15 15:12:42,649][52833] Updated weights for policy 0, policy_version 10200 (0.0009) -[2023-10-15 15:12:43,441][51532] Fps is (10 sec: 16383.1, 60 sec: 14199.3, 300 sec: 14329.1). Total num frames: 20938752. Throughput: 0: 1778.0, 1: 1764.9. Samples: 5242632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:12:43,442][51532] Avg episode reward: [(0, '13.790'), (1, '17.290')] -[2023-10-15 15:12:44,931][52866] Updated weights for policy 1, policy_version 10250 (0.0011) -[2023-10-15 15:12:45,311][52866] Updated weights for policy 1, policy_version 10260 (0.0011) -[2023-10-15 15:12:45,672][52866] Updated weights for policy 1, policy_version 10270 (0.0010) -[2023-10-15 15:12:46,482][52833] Updated weights for policy 0, policy_version 10210 (0.0010) -[2023-10-15 15:12:46,856][52833] Updated weights for policy 0, policy_version 10220 (0.0009) -[2023-10-15 15:12:47,228][52833] Updated weights for policy 0, policy_version 10230 (0.0009) -[2023-10-15 15:12:47,601][52833] Updated weights for policy 0, policy_version 10240 (0.0009) -[2023-10-15 15:12:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 21004288. Throughput: 0: 1786.3, 1: 1767.7. Samples: 5253590. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 15:12:48,442][51532] Avg episode reward: [(0, '14.780'), (1, '17.450')] -[2023-10-15 15:12:49,507][52866] Updated weights for policy 1, policy_version 10280 (0.0009) -[2023-10-15 15:12:49,875][52866] Updated weights for policy 1, policy_version 10290 (0.0010) -[2023-10-15 15:12:50,242][52866] Updated weights for policy 1, policy_version 10300 (0.0010) -[2023-10-15 15:12:51,462][52833] Updated weights for policy 0, policy_version 10250 (0.0009) -[2023-10-15 15:12:51,833][52833] Updated weights for policy 0, policy_version 10260 (0.0011) -[2023-10-15 15:12:52,199][52833] Updated weights for policy 0, policy_version 10270 (0.0010) -[2023-10-15 15:12:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 21069824. Throughput: 0: 1778.3, 1: 1763.2. Samples: 5274568. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 15:12:53,442][51532] Avg episode reward: [(0, '14.290'), (1, '17.970')] -[2023-10-15 15:12:54,181][52866] Updated weights for policy 1, policy_version 10310 (0.0009) -[2023-10-15 15:12:54,568][52866] Updated weights for policy 1, policy_version 10320 (0.0009) -[2023-10-15 15:12:54,935][52866] Updated weights for policy 1, policy_version 10330 (0.0009) -[2023-10-15 15:12:56,005][52833] Updated weights for policy 0, policy_version 10280 (0.0010) -[2023-10-15 15:12:56,376][52833] Updated weights for policy 0, policy_version 10290 (0.0010) -[2023-10-15 15:12:56,744][52833] Updated weights for policy 0, policy_version 10300 (0.0009) -[2023-10-15 15:12:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 21135360. Throughput: 0: 1759.2, 1: 1784.7. Samples: 5296080. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) -[2023-10-15 15:12:58,441][51532] Avg episode reward: [(0, '15.100'), (1, '19.180')] -[2023-10-15 15:12:58,638][52866] Updated weights for policy 1, policy_version 10340 (0.0010) -[2023-10-15 15:12:59,005][52866] Updated weights for policy 1, policy_version 10350 (0.0007) -[2023-10-15 15:12:59,382][52866] Updated weights for policy 1, policy_version 10360 (0.0009) -[2023-10-15 15:12:59,671][52518] Saving new best policy, reward=19.180! -[2023-10-15 15:13:00,586][52833] Updated weights for policy 0, policy_version 10310 (0.0009) -[2023-10-15 15:13:00,971][52833] Updated weights for policy 0, policy_version 10320 (0.0009) -[2023-10-15 15:13:01,345][52833] Updated weights for policy 0, policy_version 10330 (0.0007) -[2023-10-15 15:13:03,050][52866] Updated weights for policy 1, policy_version 10370 (0.0009) -[2023-10-15 15:13:03,412][52866] Updated weights for policy 1, policy_version 10380 (0.0007) -[2023-10-15 15:13:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 21200896. Throughput: 0: 1780.1, 1: 1761.0. Samples: 5306850. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) -[2023-10-15 15:13:03,441][51532] Avg episode reward: [(0, '15.760'), (1, '17.720')] -[2023-10-15 15:13:03,776][52866] Updated weights for policy 1, policy_version 10390 (0.0007) -[2023-10-15 15:13:04,150][52866] Updated weights for policy 1, policy_version 10400 (0.0008) -[2023-10-15 15:13:05,194][52833] Updated weights for policy 0, policy_version 10340 (0.0008) -[2023-10-15 15:13:05,571][52833] Updated weights for policy 0, policy_version 10350 (0.0007) -[2023-10-15 15:13:05,943][52833] Updated weights for policy 0, policy_version 10360 (0.0007) -[2023-10-15 15:13:07,900][52866] Updated weights for policy 1, policy_version 10410 (0.0008) -[2023-10-15 15:13:08,277][52866] Updated weights for policy 1, policy_version 10420 (0.0008) -[2023-10-15 15:13:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 21266432. Throughput: 0: 1767.7, 1: 1782.8. Samples: 5328568. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 15:13:08,442][51532] Avg episode reward: [(0, '15.070'), (1, '18.650')] -[2023-10-15 15:13:08,652][52866] Updated weights for policy 1, policy_version 10430 (0.0008) -[2023-10-15 15:13:09,425][52833] Updated weights for policy 0, policy_version 10370 (0.0007) -[2023-10-15 15:13:09,795][52833] Updated weights for policy 0, policy_version 10380 (0.0008) -[2023-10-15 15:13:10,180][52833] Updated weights for policy 0, policy_version 10390 (0.0008) -[2023-10-15 15:13:10,546][52833] Updated weights for policy 0, policy_version 10400 (0.0008) -[2023-10-15 15:13:12,390][52866] Updated weights for policy 1, policy_version 10440 (0.0008) -[2023-10-15 15:13:12,757][52866] Updated weights for policy 1, policy_version 10450 (0.0007) -[2023-10-15 15:13:13,122][52866] Updated weights for policy 1, policy_version 10460 (0.0007) -[2023-10-15 15:13:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 21364736. Throughput: 0: 1780.4, 1: 1784.0. Samples: 5350120. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 15:13:13,441][51532] Avg episode reward: [(0, '16.670'), (1, '18.630')] -[2023-10-15 15:13:14,165][52833] Updated weights for policy 0, policy_version 10410 (0.0009) -[2023-10-15 15:13:14,540][52833] Updated weights for policy 0, policy_version 10420 (0.0007) -[2023-10-15 15:13:14,908][52833] Updated weights for policy 0, policy_version 10430 (0.0008) -[2023-10-15 15:13:16,912][52866] Updated weights for policy 1, policy_version 10470 (0.0010) -[2023-10-15 15:13:17,281][52866] Updated weights for policy 1, policy_version 10480 (0.0009) -[2023-10-15 15:13:17,660][52866] Updated weights for policy 1, policy_version 10490 (0.0010) -[2023-10-15 15:13:18,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 21430272. Throughput: 0: 1781.8, 1: 1786.6. Samples: 5361080. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 15:13:18,441][51532] Avg episode reward: [(0, '15.680'), (1, '19.010')] -[2023-10-15 15:13:18,877][52833] Updated weights for policy 0, policy_version 10440 (0.0008) -[2023-10-15 15:13:19,254][52833] Updated weights for policy 0, policy_version 10450 (0.0008) -[2023-10-15 15:13:19,619][52833] Updated weights for policy 0, policy_version 10460 (0.0008) -[2023-10-15 15:13:21,287][52866] Updated weights for policy 1, policy_version 10500 (0.0010) -[2023-10-15 15:13:21,653][52866] Updated weights for policy 1, policy_version 10510 (0.0009) -[2023-10-15 15:13:22,031][52866] Updated weights for policy 1, policy_version 10520 (0.0009) -[2023-10-15 15:13:23,305][52833] Updated weights for policy 0, policy_version 10470 (0.0009) -[2023-10-15 15:13:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 21495808. Throughput: 0: 1774.8, 1: 1788.0. Samples: 5382160. Policy #0 lag: (min: 20.0, avg: 20.0, max: 23.0) -[2023-10-15 15:13:23,441][51532] Avg episode reward: [(0, '15.520'), (1, '18.320')] -[2023-10-15 15:13:23,671][52833] Updated weights for policy 0, policy_version 10480 (0.0007) -[2023-10-15 15:13:24,045][52833] Updated weights for policy 0, policy_version 10490 (0.0009) -[2023-10-15 15:13:25,790][52866] Updated weights for policy 1, policy_version 10530 (0.0008) -[2023-10-15 15:13:26,160][52866] Updated weights for policy 1, policy_version 10540 (0.0007) -[2023-10-15 15:13:26,520][52866] Updated weights for policy 1, policy_version 10550 (0.0007) -[2023-10-15 15:13:26,883][52866] Updated weights for policy 1, policy_version 10560 (0.0010) -[2023-10-15 15:13:27,817][52833] Updated weights for policy 0, policy_version 10500 (0.0008) -[2023-10-15 15:13:28,198][52833] Updated weights for policy 0, policy_version 10510 (0.0008) -[2023-10-15 15:13:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 21561344. Throughput: 0: 1806.4, 1: 1777.4. Samples: 5403904. Policy #0 lag: (min: 20.0, avg: 20.0, max: 23.0) -[2023-10-15 15:13:28,442][51532] Avg episode reward: [(0, '16.380'), (1, '18.250')] -[2023-10-15 15:13:28,566][52833] Updated weights for policy 0, policy_version 10520 (0.0009) -[2023-10-15 15:13:30,732][52866] Updated weights for policy 1, policy_version 10570 (0.0008) -[2023-10-15 15:13:31,103][52866] Updated weights for policy 1, policy_version 10580 (0.0007) -[2023-10-15 15:13:31,473][52866] Updated weights for policy 1, policy_version 10590 (0.0008) -[2023-10-15 15:13:32,386][52833] Updated weights for policy 0, policy_version 10530 (0.0009) -[2023-10-15 15:13:32,754][52833] Updated weights for policy 0, policy_version 10540 (0.0009) -[2023-10-15 15:13:33,116][52833] Updated weights for policy 0, policy_version 10550 (0.0010) -[2023-10-15 15:13:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 21626880. Throughput: 0: 1779.6, 1: 1799.5. Samples: 5414648. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) -[2023-10-15 15:13:33,441][51532] Avg episode reward: [(0, '17.640'), (1, '17.970')] -[2023-10-15 15:13:33,490][52833] Updated weights for policy 0, policy_version 10560 (0.0007) -[2023-10-15 15:13:35,128][52866] Updated weights for policy 1, policy_version 10600 (0.0008) -[2023-10-15 15:13:35,498][52866] Updated weights for policy 1, policy_version 10610 (0.0007) -[2023-10-15 15:13:35,863][52866] Updated weights for policy 1, policy_version 10620 (0.0009) -[2023-10-15 15:13:37,289][52833] Updated weights for policy 0, policy_version 10570 (0.0008) -[2023-10-15 15:13:37,657][52833] Updated weights for policy 0, policy_version 10580 (0.0010) -[2023-10-15 15:13:38,036][52833] Updated weights for policy 0, policy_version 10590 (0.0010) -[2023-10-15 15:13:38,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 21725184. Throughput: 0: 1806.9, 1: 1782.3. Samples: 5436082. Policy #0 lag: (min: 25.0, avg: 29.7, max: 57.0) -[2023-10-15 15:13:38,441][51532] Avg episode reward: [(0, '16.120'), (1, '18.660')] -[2023-10-15 15:13:39,711][52866] Updated weights for policy 1, policy_version 10630 (0.0011) -[2023-10-15 15:13:40,087][52866] Updated weights for policy 1, policy_version 10640 (0.0009) -[2023-10-15 15:13:40,456][52866] Updated weights for policy 1, policy_version 10650 (0.0007) -[2023-10-15 15:13:41,942][52833] Updated weights for policy 0, policy_version 10600 (0.0009) -[2023-10-15 15:13:42,315][52833] Updated weights for policy 0, policy_version 10610 (0.0007) -[2023-10-15 15:13:42,690][52833] Updated weights for policy 0, policy_version 10620 (0.0007) -[2023-10-15 15:13:43,441][51532] Fps is (10 sec: 16383.1, 60 sec: 14199.5, 300 sec: 14218.1). Total num frames: 21790720. Throughput: 0: 1784.7, 1: 1790.0. Samples: 5456942. Policy #0 lag: (min: 25.0, avg: 29.7, max: 57.0) -[2023-10-15 15:13:43,443][51532] Avg episode reward: [(0, '17.330'), (1, '17.590')] -[2023-10-15 15:13:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000010656_10911744.pth... -[2023-10-15 15:13:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000010624_10878976.pth... -[2023-10-15 15:13:43,494][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000008992_9207808.pth -[2023-10-15 15:13:43,495][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000008960_9175040.pth -[2023-10-15 15:13:44,156][52866] Updated weights for policy 1, policy_version 10660 (0.0010) -[2023-10-15 15:13:44,528][52866] Updated weights for policy 1, policy_version 10670 (0.0010) -[2023-10-15 15:13:44,895][52866] Updated weights for policy 1, policy_version 10680 (0.0007) -[2023-10-15 15:13:46,340][52833] Updated weights for policy 0, policy_version 10630 (0.0009) -[2023-10-15 15:13:46,728][52833] Updated weights for policy 0, policy_version 10640 (0.0010) -[2023-10-15 15:13:47,099][52833] Updated weights for policy 0, policy_version 10650 (0.0011) -[2023-10-15 15:13:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 21856256. Throughput: 0: 1799.9, 1: 1787.6. Samples: 5468288. Policy #0 lag: (min: 19.0, avg: 26.8, max: 51.0) -[2023-10-15 15:13:48,442][51532] Avg episode reward: [(0, '17.850'), (1, '17.170')] -[2023-10-15 15:13:48,673][52866] Updated weights for policy 1, policy_version 10690 (0.0008) -[2023-10-15 15:13:49,039][52866] Updated weights for policy 1, policy_version 10700 (0.0008) -[2023-10-15 15:13:49,406][52866] Updated weights for policy 1, policy_version 10710 (0.0008) -[2023-10-15 15:13:49,780][52866] Updated weights for policy 1, policy_version 10720 (0.0007) -[2023-10-15 15:13:50,807][52833] Updated weights for policy 0, policy_version 10660 (0.0010) -[2023-10-15 15:13:51,169][52833] Updated weights for policy 0, policy_version 10670 (0.0010) -[2023-10-15 15:13:51,535][52833] Updated weights for policy 0, policy_version 10680 (0.0009) -[2023-10-15 15:13:53,441][51532] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 21921792. Throughput: 0: 1784.5, 1: 1779.7. Samples: 5488952. Policy #0 lag: (min: 19.0, avg: 26.8, max: 51.0) -[2023-10-15 15:13:53,442][51532] Avg episode reward: [(0, '15.660'), (1, '18.410')] -[2023-10-15 15:13:53,653][52866] Updated weights for policy 1, policy_version 10730 (0.0007) -[2023-10-15 15:13:54,020][52866] Updated weights for policy 1, policy_version 10740 (0.0008) -[2023-10-15 15:13:54,390][52866] Updated weights for policy 1, policy_version 10750 (0.0007) -[2023-10-15 15:13:55,396][52833] Updated weights for policy 0, policy_version 10690 (0.0010) -[2023-10-15 15:13:55,758][52833] Updated weights for policy 0, policy_version 10700 (0.0007) -[2023-10-15 15:13:56,125][52833] Updated weights for policy 0, policy_version 10710 (0.0008) -[2023-10-15 15:13:56,493][52833] Updated weights for policy 0, policy_version 10720 (0.0007) -[2023-10-15 15:13:58,108][52866] Updated weights for policy 1, policy_version 10760 (0.0008) -[2023-10-15 15:13:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 21987328. Throughput: 0: 1774.1, 1: 1804.1. Samples: 5511140. Policy #0 lag: (min: 16.0, avg: 38.0, max: 48.0) -[2023-10-15 15:13:58,442][51532] Avg episode reward: [(0, '15.280'), (1, '17.210')] -[2023-10-15 15:13:58,473][52866] Updated weights for policy 1, policy_version 10770 (0.0009) -[2023-10-15 15:13:58,836][52866] Updated weights for policy 1, policy_version 10780 (0.0008) -[2023-10-15 15:14:00,365][52833] Updated weights for policy 0, policy_version 10730 (0.0009) -[2023-10-15 15:14:00,735][52833] Updated weights for policy 0, policy_version 10740 (0.0008) -[2023-10-15 15:14:01,102][52833] Updated weights for policy 0, policy_version 10750 (0.0008) -[2023-10-15 15:14:02,570][52866] Updated weights for policy 1, policy_version 10790 (0.0008) -[2023-10-15 15:14:02,941][52866] Updated weights for policy 1, policy_version 10800 (0.0009) -[2023-10-15 15:14:03,313][52866] Updated weights for policy 1, policy_version 10810 (0.0007) -[2023-10-15 15:14:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 22052864. Throughput: 0: 1782.8, 1: 1784.2. Samples: 5521596. Policy #0 lag: (min: 16.0, avg: 38.0, max: 48.0) -[2023-10-15 15:14:03,441][51532] Avg episode reward: [(0, '15.030'), (1, '18.320')] -[2023-10-15 15:14:04,876][52833] Updated weights for policy 0, policy_version 10760 (0.0011) -[2023-10-15 15:14:05,243][52833] Updated weights for policy 0, policy_version 10770 (0.0007) -[2023-10-15 15:14:05,606][52833] Updated weights for policy 0, policy_version 10780 (0.0007) -[2023-10-15 15:14:06,872][52866] Updated weights for policy 1, policy_version 10820 (0.0008) -[2023-10-15 15:14:07,238][52866] Updated weights for policy 1, policy_version 10830 (0.0007) -[2023-10-15 15:14:07,601][52866] Updated weights for policy 1, policy_version 10840 (0.0009) -[2023-10-15 15:14:08,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 22151168. Throughput: 0: 1774.8, 1: 1808.2. Samples: 5543396. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 15:14:08,442][51532] Avg episode reward: [(0, '15.380'), (1, '19.180')] -[2023-10-15 15:14:09,446][52833] Updated weights for policy 0, policy_version 10790 (0.0008) -[2023-10-15 15:14:09,821][52833] Updated weights for policy 0, policy_version 10800 (0.0009) -[2023-10-15 15:14:10,197][52833] Updated weights for policy 0, policy_version 10810 (0.0007) -[2023-10-15 15:14:11,365][52866] Updated weights for policy 1, policy_version 10850 (0.0008) -[2023-10-15 15:14:11,733][52866] Updated weights for policy 1, policy_version 10860 (0.0009) -[2023-10-15 15:14:12,097][52866] Updated weights for policy 1, policy_version 10870 (0.0008) -[2023-10-15 15:14:12,461][52866] Updated weights for policy 1, policy_version 10880 (0.0009) -[2023-10-15 15:14:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 22216704. Throughput: 0: 1779.9, 1: 1794.0. Samples: 5564728. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 15:14:13,441][51532] Avg episode reward: [(0, '15.290'), (1, '18.590')] -[2023-10-15 15:14:13,810][52833] Updated weights for policy 0, policy_version 10820 (0.0008) -[2023-10-15 15:14:14,172][52833] Updated weights for policy 0, policy_version 10830 (0.0010) -[2023-10-15 15:14:14,539][52833] Updated weights for policy 0, policy_version 10840 (0.0010) -[2023-10-15 15:14:16,376][52866] Updated weights for policy 1, policy_version 10890 (0.0010) -[2023-10-15 15:14:16,756][52866] Updated weights for policy 1, policy_version 10900 (0.0008) -[2023-10-15 15:14:17,126][52866] Updated weights for policy 1, policy_version 10910 (0.0009) -[2023-10-15 15:14:18,264][52833] Updated weights for policy 0, policy_version 10850 (0.0009) -[2023-10-15 15:14:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 22282240. Throughput: 0: 1777.2, 1: 1806.4. Samples: 5575906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:14:18,441][51532] Avg episode reward: [(0, '15.860'), (1, '18.510')] -[2023-10-15 15:14:18,637][52833] Updated weights for policy 0, policy_version 10860 (0.0007) -[2023-10-15 15:14:19,003][52833] Updated weights for policy 0, policy_version 10870 (0.0007) -[2023-10-15 15:14:19,368][52833] Updated weights for policy 0, policy_version 10880 (0.0007) -[2023-10-15 15:14:21,013][52866] Updated weights for policy 1, policy_version 10920 (0.0008) -[2023-10-15 15:14:21,377][52866] Updated weights for policy 1, policy_version 10930 (0.0009) -[2023-10-15 15:14:21,752][52866] Updated weights for policy 1, policy_version 10940 (0.0007) -[2023-10-15 15:14:22,920][52833] Updated weights for policy 0, policy_version 10890 (0.0010) -[2023-10-15 15:14:23,293][52833] Updated weights for policy 0, policy_version 10900 (0.0008) -[2023-10-15 15:14:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 22347776. Throughput: 0: 1784.0, 1: 1789.5. Samples: 5596894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:14:23,442][51532] Avg episode reward: [(0, '16.820'), (1, '18.810')] -[2023-10-15 15:14:23,663][52833] Updated weights for policy 0, policy_version 10910 (0.0011) -[2023-10-15 15:14:25,604][52866] Updated weights for policy 1, policy_version 10950 (0.0007) -[2023-10-15 15:14:25,980][52866] Updated weights for policy 1, policy_version 10960 (0.0008) -[2023-10-15 15:14:26,351][52866] Updated weights for policy 1, policy_version 10970 (0.0010) -[2023-10-15 15:14:27,507][52833] Updated weights for policy 0, policy_version 10920 (0.0010) -[2023-10-15 15:14:27,872][52833] Updated weights for policy 0, policy_version 10930 (0.0012) -[2023-10-15 15:14:28,251][52833] Updated weights for policy 0, policy_version 10940 (0.0007) -[2023-10-15 15:14:28,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 22446080. Throughput: 0: 1805.3, 1: 1781.3. Samples: 5618334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:14:28,442][51532] Avg episode reward: [(0, '16.720'), (1, '18.220')] -[2023-10-15 15:14:30,073][52866] Updated weights for policy 1, policy_version 10980 (0.0010) -[2023-10-15 15:14:30,446][52866] Updated weights for policy 1, policy_version 10990 (0.0009) -[2023-10-15 15:14:30,818][52866] Updated weights for policy 1, policy_version 11000 (0.0008) -[2023-10-15 15:14:32,032][52833] Updated weights for policy 0, policy_version 10950 (0.0008) -[2023-10-15 15:14:32,408][52833] Updated weights for policy 0, policy_version 10960 (0.0007) -[2023-10-15 15:14:32,783][52833] Updated weights for policy 0, policy_version 10970 (0.0007) -[2023-10-15 15:14:33,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 22511616. Throughput: 0: 1786.8, 1: 1787.5. Samples: 5629132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:14:33,441][51532] Avg episode reward: [(0, '16.530'), (1, '17.530')] -[2023-10-15 15:14:34,591][52866] Updated weights for policy 1, policy_version 11010 (0.0008) -[2023-10-15 15:14:34,967][52866] Updated weights for policy 1, policy_version 11020 (0.0009) -[2023-10-15 15:14:35,327][52866] Updated weights for policy 1, policy_version 11030 (0.0008) -[2023-10-15 15:14:35,692][52866] Updated weights for policy 1, policy_version 11040 (0.0009) -[2023-10-15 15:14:36,521][52833] Updated weights for policy 0, policy_version 10980 (0.0008) -[2023-10-15 15:14:36,891][52833] Updated weights for policy 0, policy_version 10990 (0.0009) -[2023-10-15 15:14:37,262][52833] Updated weights for policy 0, policy_version 11000 (0.0008) -[2023-10-15 15:14:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 22577152. Throughput: 0: 1805.4, 1: 1784.0. Samples: 5650478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:14:38,442][51532] Avg episode reward: [(0, '18.010'), (1, '18.270')] -[2023-10-15 15:14:39,297][52866] Updated weights for policy 1, policy_version 11050 (0.0007) -[2023-10-15 15:14:39,657][52866] Updated weights for policy 1, policy_version 11060 (0.0007) -[2023-10-15 15:14:40,035][52866] Updated weights for policy 1, policy_version 11070 (0.0008) -[2023-10-15 15:14:40,993][52833] Updated weights for policy 0, policy_version 11010 (0.0009) -[2023-10-15 15:14:41,368][52833] Updated weights for policy 0, policy_version 11020 (0.0010) -[2023-10-15 15:14:41,739][52833] Updated weights for policy 0, policy_version 11030 (0.0010) -[2023-10-15 15:14:42,103][52833] Updated weights for policy 0, policy_version 11040 (0.0008) -[2023-10-15 15:14:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 22642688. Throughput: 0: 1787.1, 1: 1790.5. Samples: 5672132. Policy #0 lag: (min: 26.0, avg: 35.9, max: 58.0) -[2023-10-15 15:14:43,443][51532] Avg episode reward: [(0, '17.260'), (1, '18.120')] -[2023-10-15 15:14:43,818][52866] Updated weights for policy 1, policy_version 11080 (0.0008) -[2023-10-15 15:14:44,178][52866] Updated weights for policy 1, policy_version 11090 (0.0008) -[2023-10-15 15:14:44,555][52866] Updated weights for policy 1, policy_version 11100 (0.0008) -[2023-10-15 15:14:46,017][52833] Updated weights for policy 0, policy_version 11050 (0.0009) -[2023-10-15 15:14:46,381][52833] Updated weights for policy 0, policy_version 11060 (0.0007) -[2023-10-15 15:14:46,763][52833] Updated weights for policy 0, policy_version 11070 (0.0008) -[2023-10-15 15:14:48,401][52866] Updated weights for policy 1, policy_version 11110 (0.0009) -[2023-10-15 15:14:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 22708224. Throughput: 0: 1803.5, 1: 1782.4. Samples: 5682958. Policy #0 lag: (min: 26.0, avg: 35.9, max: 58.0) -[2023-10-15 15:14:48,441][51532] Avg episode reward: [(0, '17.490'), (1, '18.420')] -[2023-10-15 15:14:48,767][52866] Updated weights for policy 1, policy_version 11120 (0.0009) -[2023-10-15 15:14:49,134][52866] Updated weights for policy 1, policy_version 11130 (0.0007) -[2023-10-15 15:14:50,689][52833] Updated weights for policy 0, policy_version 11080 (0.0007) -[2023-10-15 15:14:51,054][52833] Updated weights for policy 0, policy_version 11090 (0.0007) -[2023-10-15 15:14:51,425][52833] Updated weights for policy 0, policy_version 11100 (0.0008) -[2023-10-15 15:14:52,944][52866] Updated weights for policy 1, policy_version 11140 (0.0008) -[2023-10-15 15:14:53,307][52866] Updated weights for policy 1, policy_version 11150 (0.0008) -[2023-10-15 15:14:53,441][51532] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 22773760. Throughput: 0: 1785.4, 1: 1781.0. Samples: 5703886. Policy #0 lag: (min: 0.0, avg: 26.8, max: 32.0) -[2023-10-15 15:14:53,441][51532] Avg episode reward: [(0, '17.290'), (1, '18.670')] -[2023-10-15 15:14:53,673][52866] Updated weights for policy 1, policy_version 11160 (0.0008) -[2023-10-15 15:14:55,126][52833] Updated weights for policy 0, policy_version 11110 (0.0007) -[2023-10-15 15:14:55,498][52833] Updated weights for policy 0, policy_version 11120 (0.0009) -[2023-10-15 15:14:55,869][52833] Updated weights for policy 0, policy_version 11130 (0.0007) -[2023-10-15 15:14:57,247][52866] Updated weights for policy 1, policy_version 11170 (0.0008) -[2023-10-15 15:14:57,606][52866] Updated weights for policy 1, policy_version 11180 (0.0008) -[2023-10-15 15:14:57,986][52866] Updated weights for policy 1, policy_version 11190 (0.0007) -[2023-10-15 15:14:58,343][52866] Updated weights for policy 1, policy_version 11200 (0.0007) -[2023-10-15 15:14:58,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 22872064. Throughput: 0: 1786.3, 1: 1783.6. Samples: 5725372. Policy #0 lag: (min: 0.0, avg: 26.8, max: 32.0) -[2023-10-15 15:14:58,442][51532] Avg episode reward: [(0, '18.070'), (1, '19.420')] -[2023-10-15 15:14:58,451][52518] Saving new best policy, reward=19.420! -[2023-10-15 15:14:59,599][52833] Updated weights for policy 0, policy_version 11140 (0.0008) -[2023-10-15 15:14:59,958][52833] Updated weights for policy 0, policy_version 11150 (0.0007) -[2023-10-15 15:15:00,335][52833] Updated weights for policy 0, policy_version 11160 (0.0009) -[2023-10-15 15:15:02,150][52866] Updated weights for policy 1, policy_version 11210 (0.0008) -[2023-10-15 15:15:02,520][52866] Updated weights for policy 1, policy_version 11220 (0.0007) -[2023-10-15 15:15:02,882][52866] Updated weights for policy 1, policy_version 11230 (0.0008) -[2023-10-15 15:15:03,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 22937600. Throughput: 0: 1786.5, 1: 1773.7. Samples: 5736116. Policy #0 lag: (min: 29.0, avg: 30.0, max: 51.0) -[2023-10-15 15:15:03,442][51532] Avg episode reward: [(0, '17.730'), (1, '18.230')] -[2023-10-15 15:15:04,095][52833] Updated weights for policy 0, policy_version 11170 (0.0008) -[2023-10-15 15:15:04,466][52833] Updated weights for policy 0, policy_version 11180 (0.0009) -[2023-10-15 15:15:04,830][52833] Updated weights for policy 0, policy_version 11190 (0.0007) -[2023-10-15 15:15:05,204][52833] Updated weights for policy 0, policy_version 11200 (0.0010) -[2023-10-15 15:15:06,732][52866] Updated weights for policy 1, policy_version 11240 (0.0008) -[2023-10-15 15:15:07,107][52866] Updated weights for policy 1, policy_version 11250 (0.0010) -[2023-10-15 15:15:07,463][52866] Updated weights for policy 1, policy_version 11260 (0.0010) -[2023-10-15 15:15:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 23003136. Throughput: 0: 1783.8, 1: 1794.2. Samples: 5757904. Policy #0 lag: (min: 29.0, avg: 30.0, max: 51.0) -[2023-10-15 15:15:08,442][51532] Avg episode reward: [(0, '17.760'), (1, '18.080')] -[2023-10-15 15:15:09,006][52833] Updated weights for policy 0, policy_version 11210 (0.0009) -[2023-10-15 15:15:09,377][52833] Updated weights for policy 0, policy_version 11220 (0.0007) -[2023-10-15 15:15:09,748][52833] Updated weights for policy 0, policy_version 11230 (0.0010) -[2023-10-15 15:15:11,350][52866] Updated weights for policy 1, policy_version 11270 (0.0009) -[2023-10-15 15:15:11,728][52866] Updated weights for policy 1, policy_version 11280 (0.0008) -[2023-10-15 15:15:12,095][52866] Updated weights for policy 1, policy_version 11290 (0.0010) -[2023-10-15 15:15:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 23068672. Throughput: 0: 1793.4, 1: 1780.0. Samples: 5779134. Policy #0 lag: (min: 29.0, avg: 30.0, max: 51.0) -[2023-10-15 15:15:13,442][51532] Avg episode reward: [(0, '18.140'), (1, '17.300')] -[2023-10-15 15:15:13,634][52833] Updated weights for policy 0, policy_version 11240 (0.0008) -[2023-10-15 15:15:14,001][52833] Updated weights for policy 0, policy_version 11250 (0.0007) -[2023-10-15 15:15:14,377][52833] Updated weights for policy 0, policy_version 11260 (0.0008) -[2023-10-15 15:15:15,772][52866] Updated weights for policy 1, policy_version 11300 (0.0008) -[2023-10-15 15:15:16,142][52866] Updated weights for policy 1, policy_version 11310 (0.0007) -[2023-10-15 15:15:16,502][52866] Updated weights for policy 1, policy_version 11320 (0.0008) -[2023-10-15 15:15:18,319][52833] Updated weights for policy 0, policy_version 11270 (0.0008) -[2023-10-15 15:15:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 23134208. Throughput: 0: 1772.2, 1: 1802.4. Samples: 5789988. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:15:18,443][51532] Avg episode reward: [(0, '18.170'), (1, '18.320')] -[2023-10-15 15:15:18,709][52833] Updated weights for policy 0, policy_version 11280 (0.0008) -[2023-10-15 15:15:19,071][52833] Updated weights for policy 0, policy_version 11290 (0.0007) -[2023-10-15 15:15:20,269][52866] Updated weights for policy 1, policy_version 11330 (0.0010) -[2023-10-15 15:15:20,634][52866] Updated weights for policy 1, policy_version 11340 (0.0007) -[2023-10-15 15:15:21,004][52866] Updated weights for policy 1, policy_version 11350 (0.0008) -[2023-10-15 15:15:21,366][52866] Updated weights for policy 1, policy_version 11360 (0.0010) -[2023-10-15 15:15:22,626][52833] Updated weights for policy 0, policy_version 11300 (0.0010) -[2023-10-15 15:15:22,994][52833] Updated weights for policy 0, policy_version 11310 (0.0009) -[2023-10-15 15:15:23,364][52833] Updated weights for policy 0, policy_version 11320 (0.0010) -[2023-10-15 15:15:23,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 23199744. Throughput: 0: 1783.6, 1: 1786.3. Samples: 5811120. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:15:23,441][51532] Avg episode reward: [(0, '17.400'), (1, '17.660')] -[2023-10-15 15:15:25,229][52866] Updated weights for policy 1, policy_version 11370 (0.0008) -[2023-10-15 15:15:25,603][52866] Updated weights for policy 1, policy_version 11380 (0.0009) -[2023-10-15 15:15:25,970][52866] Updated weights for policy 1, policy_version 11390 (0.0007) -[2023-10-15 15:15:27,082][52833] Updated weights for policy 0, policy_version 11330 (0.0010) -[2023-10-15 15:15:27,439][52833] Updated weights for policy 0, policy_version 11340 (0.0007) -[2023-10-15 15:15:27,812][52833] Updated weights for policy 0, policy_version 11350 (0.0007) -[2023-10-15 15:15:28,176][52833] Updated weights for policy 0, policy_version 11360 (0.0007) -[2023-10-15 15:15:28,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 23298048. Throughput: 0: 1785.7, 1: 1772.9. Samples: 5832268. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:15:28,442][51532] Avg episode reward: [(0, '17.540'), (1, '18.400')] -[2023-10-15 15:15:29,783][52866] Updated weights for policy 1, policy_version 11400 (0.0009) -[2023-10-15 15:15:30,150][52866] Updated weights for policy 1, policy_version 11410 (0.0007) -[2023-10-15 15:15:30,514][52866] Updated weights for policy 1, policy_version 11420 (0.0009) -[2023-10-15 15:15:31,849][52833] Updated weights for policy 0, policy_version 11370 (0.0008) -[2023-10-15 15:15:32,227][52833] Updated weights for policy 0, policy_version 11380 (0.0010) -[2023-10-15 15:15:32,605][52833] Updated weights for policy 0, policy_version 11390 (0.0009) -[2023-10-15 15:15:33,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 23363584. Throughput: 0: 1782.3, 1: 1775.8. Samples: 5843074. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:15:33,442][51532] Avg episode reward: [(0, '16.830'), (1, '17.870')] -[2023-10-15 15:15:34,266][52866] Updated weights for policy 1, policy_version 11430 (0.0011) -[2023-10-15 15:15:34,633][52866] Updated weights for policy 1, policy_version 11440 (0.0010) -[2023-10-15 15:15:35,010][52866] Updated weights for policy 1, policy_version 11450 (0.0010) -[2023-10-15 15:15:36,278][52833] Updated weights for policy 0, policy_version 11400 (0.0009) -[2023-10-15 15:15:36,645][52833] Updated weights for policy 0, policy_version 11410 (0.0010) -[2023-10-15 15:15:37,020][52833] Updated weights for policy 0, policy_version 11420 (0.0009) -[2023-10-15 15:15:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 23429120. Throughput: 0: 1791.7, 1: 1779.5. Samples: 5864588. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:15:38,442][51532] Avg episode reward: [(0, '18.390'), (1, '18.560')] -[2023-10-15 15:15:38,648][52866] Updated weights for policy 1, policy_version 11460 (0.0008) -[2023-10-15 15:15:39,019][52866] Updated weights for policy 1, policy_version 11470 (0.0007) -[2023-10-15 15:15:39,384][52866] Updated weights for policy 1, policy_version 11480 (0.0007) -[2023-10-15 15:15:40,801][52833] Updated weights for policy 0, policy_version 11430 (0.0010) -[2023-10-15 15:15:41,177][52833] Updated weights for policy 0, policy_version 11440 (0.0010) -[2023-10-15 15:15:41,539][52833] Updated weights for policy 0, policy_version 11450 (0.0009) -[2023-10-15 15:15:43,121][52866] Updated weights for policy 1, policy_version 11490 (0.0008) -[2023-10-15 15:15:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 23494656. Throughput: 0: 1782.1, 1: 1805.0. Samples: 5886792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:15:43,442][51532] Avg episode reward: [(0, '17.640'), (1, '18.560')] -[2023-10-15 15:15:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000011456_11730944.pth... -[2023-10-15 15:15:43,486][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000009792_10027008.pth -[2023-10-15 15:15:43,490][52866] Updated weights for policy 1, policy_version 11500 (0.0009) -[2023-10-15 15:15:43,857][52866] Updated weights for policy 1, policy_version 11510 (0.0008) -[2023-10-15 15:15:44,227][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000011520_11796480.pth... -[2023-10-15 15:15:44,227][52866] Updated weights for policy 1, policy_version 11520 (0.0009) -[2023-10-15 15:15:44,265][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000009824_10059776.pth -[2023-10-15 15:15:45,462][52833] Updated weights for policy 0, policy_version 11460 (0.0007) -[2023-10-15 15:15:45,831][52833] Updated weights for policy 0, policy_version 11470 (0.0007) -[2023-10-15 15:15:46,200][52833] Updated weights for policy 0, policy_version 11480 (0.0008) -[2023-10-15 15:15:48,012][52866] Updated weights for policy 1, policy_version 11530 (0.0010) -[2023-10-15 15:15:48,380][52866] Updated weights for policy 1, policy_version 11540 (0.0008) -[2023-10-15 15:15:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 23560192. Throughput: 0: 1798.6, 1: 1784.2. Samples: 5897342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:15:48,441][51532] Avg episode reward: [(0, '17.910'), (1, '20.080')] -[2023-10-15 15:15:48,749][52866] Updated weights for policy 1, policy_version 11550 (0.0009) -[2023-10-15 15:15:48,824][52518] Saving new best policy, reward=20.080! -[2023-10-15 15:15:50,024][52833] Updated weights for policy 0, policy_version 11490 (0.0007) -[2023-10-15 15:15:50,397][52833] Updated weights for policy 0, policy_version 11500 (0.0009) -[2023-10-15 15:15:50,766][52833] Updated weights for policy 0, policy_version 11510 (0.0009) -[2023-10-15 15:15:51,139][52833] Updated weights for policy 0, policy_version 11520 (0.0010) -[2023-10-15 15:15:52,459][52866] Updated weights for policy 1, policy_version 11560 (0.0008) -[2023-10-15 15:15:52,823][52866] Updated weights for policy 1, policy_version 11570 (0.0008) -[2023-10-15 15:15:53,200][52866] Updated weights for policy 1, policy_version 11580 (0.0008) -[2023-10-15 15:15:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 23658496. Throughput: 0: 1775.3, 1: 1802.9. Samples: 5918922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:15:53,441][51532] Avg episode reward: [(0, '18.300'), (1, '18.880')] -[2023-10-15 15:15:54,945][52833] Updated weights for policy 0, policy_version 11530 (0.0007) -[2023-10-15 15:15:55,316][52833] Updated weights for policy 0, policy_version 11540 (0.0007) -[2023-10-15 15:15:55,682][52833] Updated weights for policy 0, policy_version 11550 (0.0008) -[2023-10-15 15:15:57,023][52866] Updated weights for policy 1, policy_version 11590 (0.0007) -[2023-10-15 15:15:57,396][52866] Updated weights for policy 1, policy_version 11600 (0.0010) -[2023-10-15 15:15:57,760][52866] Updated weights for policy 1, policy_version 11610 (0.0011) -[2023-10-15 15:15:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 23724032. Throughput: 0: 1778.8, 1: 1792.6. Samples: 5939848. Policy #0 lag: (min: 31.0, avg: 46.6, max: 63.0) -[2023-10-15 15:15:58,442][51532] Avg episode reward: [(0, '17.490'), (1, '19.670')] -[2023-10-15 15:15:59,368][52833] Updated weights for policy 0, policy_version 11560 (0.0007) -[2023-10-15 15:15:59,731][52833] Updated weights for policy 0, policy_version 11570 (0.0007) -[2023-10-15 15:16:00,107][52833] Updated weights for policy 0, policy_version 11580 (0.0008) -[2023-10-15 15:16:01,552][52866] Updated weights for policy 1, policy_version 11620 (0.0011) -[2023-10-15 15:16:01,921][52866] Updated weights for policy 1, policy_version 11630 (0.0008) -[2023-10-15 15:16:02,292][52866] Updated weights for policy 1, policy_version 11640 (0.0010) -[2023-10-15 15:16:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 23789568. Throughput: 0: 1784.1, 1: 1797.2. Samples: 5951148. Policy #0 lag: (min: 31.0, avg: 46.6, max: 63.0) -[2023-10-15 15:16:03,441][51532] Avg episode reward: [(0, '16.730'), (1, '20.360')] -[2023-10-15 15:16:03,442][52518] Saving new best policy, reward=20.360! -[2023-10-15 15:16:04,043][52833] Updated weights for policy 0, policy_version 11590 (0.0008) -[2023-10-15 15:16:04,424][52833] Updated weights for policy 0, policy_version 11600 (0.0008) -[2023-10-15 15:16:04,796][52833] Updated weights for policy 0, policy_version 11610 (0.0007) -[2023-10-15 15:16:06,002][52866] Updated weights for policy 1, policy_version 11650 (0.0008) -[2023-10-15 15:16:06,362][52866] Updated weights for policy 1, policy_version 11660 (0.0007) -[2023-10-15 15:16:06,738][52866] Updated weights for policy 1, policy_version 11670 (0.0008) -[2023-10-15 15:16:07,113][52866] Updated weights for policy 1, policy_version 11680 (0.0010) -[2023-10-15 15:16:08,344][52833] Updated weights for policy 0, policy_version 11620 (0.0010) -[2023-10-15 15:16:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 23855104. Throughput: 0: 1782.3, 1: 1792.0. Samples: 5971962. Policy #0 lag: (min: 31.0, avg: 46.6, max: 63.0) -[2023-10-15 15:16:08,442][51532] Avg episode reward: [(0, '16.950'), (1, '19.690')] -[2023-10-15 15:16:08,707][52833] Updated weights for policy 0, policy_version 11630 (0.0008) -[2023-10-15 15:16:09,077][52833] Updated weights for policy 0, policy_version 11640 (0.0007) -[2023-10-15 15:16:10,853][52866] Updated weights for policy 1, policy_version 11690 (0.0008) -[2023-10-15 15:16:11,219][52866] Updated weights for policy 1, policy_version 11700 (0.0007) -[2023-10-15 15:16:11,583][52866] Updated weights for policy 1, policy_version 11710 (0.0010) -[2023-10-15 15:16:12,939][52833] Updated weights for policy 0, policy_version 11650 (0.0009) -[2023-10-15 15:16:13,311][52833] Updated weights for policy 0, policy_version 11660 (0.0008) -[2023-10-15 15:16:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 23920640. Throughput: 0: 1800.4, 1: 1795.8. Samples: 5994100. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) -[2023-10-15 15:16:13,442][51532] Avg episode reward: [(0, '18.150'), (1, '19.180')] -[2023-10-15 15:16:13,674][52833] Updated weights for policy 0, policy_version 11670 (0.0007) -[2023-10-15 15:16:14,044][52833] Updated weights for policy 0, policy_version 11680 (0.0009) -[2023-10-15 15:16:15,264][52866] Updated weights for policy 1, policy_version 11720 (0.0008) -[2023-10-15 15:16:15,637][52866] Updated weights for policy 1, policy_version 11730 (0.0007) -[2023-10-15 15:16:15,994][52866] Updated weights for policy 1, policy_version 11740 (0.0008) -[2023-10-15 15:16:17,793][52833] Updated weights for policy 0, policy_version 11690 (0.0009) -[2023-10-15 15:16:18,153][52833] Updated weights for policy 0, policy_version 11700 (0.0009) -[2023-10-15 15:16:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 23986176. Throughput: 0: 1776.8, 1: 1803.1. Samples: 6004172. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) -[2023-10-15 15:16:18,442][51532] Avg episode reward: [(0, '16.770'), (1, '20.270')] -[2023-10-15 15:16:18,526][52833] Updated weights for policy 0, policy_version 11710 (0.0007) -[2023-10-15 15:16:19,730][52866] Updated weights for policy 1, policy_version 11750 (0.0010) -[2023-10-15 15:16:20,100][52866] Updated weights for policy 1, policy_version 11760 (0.0010) -[2023-10-15 15:16:20,459][52866] Updated weights for policy 1, policy_version 11770 (0.0009) -[2023-10-15 15:16:22,414][52833] Updated weights for policy 0, policy_version 11720 (0.0007) -[2023-10-15 15:16:22,784][52833] Updated weights for policy 0, policy_version 11730 (0.0009) -[2023-10-15 15:16:23,165][52833] Updated weights for policy 0, policy_version 11740 (0.0010) -[2023-10-15 15:16:23,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 24084480. Throughput: 0: 1795.6, 1: 1798.6. Samples: 6026324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:16:23,441][51532] Avg episode reward: [(0, '17.180'), (1, '20.030')] -[2023-10-15 15:16:24,094][52866] Updated weights for policy 1, policy_version 11780 (0.0010) -[2023-10-15 15:16:24,460][52866] Updated weights for policy 1, policy_version 11790 (0.0011) -[2023-10-15 15:16:24,824][52866] Updated weights for policy 1, policy_version 11800 (0.0010) -[2023-10-15 15:16:26,885][52833] Updated weights for policy 0, policy_version 11750 (0.0009) -[2023-10-15 15:16:27,253][52833] Updated weights for policy 0, policy_version 11760 (0.0008) -[2023-10-15 15:16:27,627][52833] Updated weights for policy 0, policy_version 11770 (0.0008) -[2023-10-15 15:16:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 24150016. Throughput: 0: 1767.5, 1: 1800.1. Samples: 6047338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:16:28,442][51532] Avg episode reward: [(0, '18.560'), (1, '21.180')] -[2023-10-15 15:16:28,455][52518] Saving new best policy, reward=21.180! -[2023-10-15 15:16:28,733][52866] Updated weights for policy 1, policy_version 11810 (0.0008) -[2023-10-15 15:16:29,110][52866] Updated weights for policy 1, policy_version 11820 (0.0011) -[2023-10-15 15:16:29,477][52866] Updated weights for policy 1, policy_version 11830 (0.0010) -[2023-10-15 15:16:29,843][52866] Updated weights for policy 1, policy_version 11840 (0.0008) -[2023-10-15 15:16:31,384][52833] Updated weights for policy 0, policy_version 11780 (0.0010) -[2023-10-15 15:16:31,756][52833] Updated weights for policy 0, policy_version 11790 (0.0009) -[2023-10-15 15:16:32,132][52833] Updated weights for policy 0, policy_version 11800 (0.0008) -[2023-10-15 15:16:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 24215552. Throughput: 0: 1788.5, 1: 1797.1. Samples: 6058692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:16:33,441][51532] Avg episode reward: [(0, '19.090'), (1, '21.600')] -[2023-10-15 15:16:33,442][52410] Saving new best policy, reward=19.090! -[2023-10-15 15:16:33,453][52866] Updated weights for policy 1, policy_version 11850 (0.0009) -[2023-10-15 15:16:33,815][52866] Updated weights for policy 1, policy_version 11860 (0.0011) -[2023-10-15 15:16:34,185][52866] Updated weights for policy 1, policy_version 11870 (0.0007) -[2023-10-15 15:16:34,256][52518] Saving new best policy, reward=21.600! -[2023-10-15 15:16:35,900][52833] Updated weights for policy 0, policy_version 11810 (0.0008) -[2023-10-15 15:16:36,266][52833] Updated weights for policy 0, policy_version 11820 (0.0010) -[2023-10-15 15:16:36,639][52833] Updated weights for policy 0, policy_version 11830 (0.0009) -[2023-10-15 15:16:37,009][52833] Updated weights for policy 0, policy_version 11840 (0.0010) -[2023-10-15 15:16:37,923][52866] Updated weights for policy 1, policy_version 11880 (0.0010) -[2023-10-15 15:16:38,294][52866] Updated weights for policy 1, policy_version 11890 (0.0009) -[2023-10-15 15:16:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 24281088. Throughput: 0: 1782.0, 1: 1799.1. Samples: 6080072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:16:38,441][51532] Avg episode reward: [(0, '18.650'), (1, '21.350')] -[2023-10-15 15:16:38,659][52866] Updated weights for policy 1, policy_version 11900 (0.0009) -[2023-10-15 15:16:40,830][52833] Updated weights for policy 0, policy_version 11850 (0.0008) -[2023-10-15 15:16:41,208][52833] Updated weights for policy 0, policy_version 11860 (0.0010) -[2023-10-15 15:16:41,586][52833] Updated weights for policy 0, policy_version 11870 (0.0011) -[2023-10-15 15:16:42,522][52866] Updated weights for policy 1, policy_version 11910 (0.0008) -[2023-10-15 15:16:42,906][52866] Updated weights for policy 1, policy_version 11920 (0.0007) -[2023-10-15 15:16:43,272][52866] Updated weights for policy 1, policy_version 11930 (0.0007) -[2023-10-15 15:16:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 24346624. Throughput: 0: 1775.6, 1: 1810.3. Samples: 6101214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:16:43,441][51532] Avg episode reward: [(0, '19.400'), (1, '21.330')] -[2023-10-15 15:16:43,451][52410] Saving new best policy, reward=19.400! -[2023-10-15 15:16:45,363][52833] Updated weights for policy 0, policy_version 11880 (0.0008) -[2023-10-15 15:16:45,736][52833] Updated weights for policy 0, policy_version 11890 (0.0008) -[2023-10-15 15:16:46,107][52833] Updated weights for policy 0, policy_version 11900 (0.0007) -[2023-10-15 15:16:46,986][52866] Updated weights for policy 1, policy_version 11940 (0.0009) -[2023-10-15 15:16:47,367][52866] Updated weights for policy 1, policy_version 11950 (0.0009) -[2023-10-15 15:16:47,728][52866] Updated weights for policy 1, policy_version 11960 (0.0008) -[2023-10-15 15:16:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 24444928. Throughput: 0: 1788.2, 1: 1800.2. Samples: 6112626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:16:48,441][51532] Avg episode reward: [(0, '18.510'), (1, '20.650')] -[2023-10-15 15:16:49,899][52833] Updated weights for policy 0, policy_version 11910 (0.0008) -[2023-10-15 15:16:50,267][52833] Updated weights for policy 0, policy_version 11920 (0.0010) -[2023-10-15 15:16:50,636][52833] Updated weights for policy 0, policy_version 11930 (0.0007) -[2023-10-15 15:16:51,500][52866] Updated weights for policy 1, policy_version 11970 (0.0008) -[2023-10-15 15:16:51,869][52866] Updated weights for policy 1, policy_version 11980 (0.0008) -[2023-10-15 15:16:52,245][52866] Updated weights for policy 1, policy_version 11990 (0.0009) -[2023-10-15 15:16:52,609][52866] Updated weights for policy 1, policy_version 12000 (0.0008) -[2023-10-15 15:16:53,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 24510464. Throughput: 0: 1783.9, 1: 1809.9. Samples: 6133684. Policy #0 lag: (min: 1.0, avg: 21.5, max: 33.0) -[2023-10-15 15:16:53,442][51532] Avg episode reward: [(0, '17.760'), (1, '20.850')] -[2023-10-15 15:16:54,292][52833] Updated weights for policy 0, policy_version 11940 (0.0009) -[2023-10-15 15:16:54,663][52833] Updated weights for policy 0, policy_version 11950 (0.0010) -[2023-10-15 15:16:55,036][52833] Updated weights for policy 0, policy_version 11960 (0.0008) -[2023-10-15 15:16:56,325][52866] Updated weights for policy 1, policy_version 12010 (0.0008) -[2023-10-15 15:16:56,691][52866] Updated weights for policy 1, policy_version 12020 (0.0008) -[2023-10-15 15:16:57,051][52866] Updated weights for policy 1, policy_version 12030 (0.0010) -[2023-10-15 15:16:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 24576000. Throughput: 0: 1785.6, 1: 1799.4. Samples: 6155424. Policy #0 lag: (min: 1.0, avg: 21.5, max: 33.0) -[2023-10-15 15:16:58,441][51532] Avg episode reward: [(0, '17.380'), (1, '19.030')] -[2023-10-15 15:16:58,953][52833] Updated weights for policy 0, policy_version 11970 (0.0010) -[2023-10-15 15:16:59,328][52833] Updated weights for policy 0, policy_version 11980 (0.0009) -[2023-10-15 15:16:59,695][52833] Updated weights for policy 0, policy_version 11990 (0.0007) -[2023-10-15 15:17:00,062][52833] Updated weights for policy 0, policy_version 12000 (0.0010) -[2023-10-15 15:17:00,777][52866] Updated weights for policy 1, policy_version 12040 (0.0008) -[2023-10-15 15:17:01,150][52866] Updated weights for policy 1, policy_version 12050 (0.0011) -[2023-10-15 15:17:01,508][52866] Updated weights for policy 1, policy_version 12060 (0.0009) -[2023-10-15 15:17:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 24641536. Throughput: 0: 1783.1, 1: 1812.1. Samples: 6165958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:03,442][51532] Avg episode reward: [(0, '17.920'), (1, '19.090')] -[2023-10-15 15:17:03,879][52833] Updated weights for policy 0, policy_version 12010 (0.0008) -[2023-10-15 15:17:04,243][52833] Updated weights for policy 0, policy_version 12020 (0.0008) -[2023-10-15 15:17:04,619][52833] Updated weights for policy 0, policy_version 12030 (0.0008) -[2023-10-15 15:17:05,081][52866] Updated weights for policy 1, policy_version 12070 (0.0009) -[2023-10-15 15:17:05,451][52866] Updated weights for policy 1, policy_version 12080 (0.0007) -[2023-10-15 15:17:05,828][52866] Updated weights for policy 1, policy_version 12090 (0.0007) -[2023-10-15 15:17:08,258][52833] Updated weights for policy 0, policy_version 12040 (0.0009) -[2023-10-15 15:17:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 24707072. Throughput: 0: 1781.5, 1: 1800.0. Samples: 6187496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:08,442][51532] Avg episode reward: [(0, '18.290'), (1, '20.180')] -[2023-10-15 15:17:08,637][52833] Updated weights for policy 0, policy_version 12050 (0.0009) -[2023-10-15 15:17:09,001][52833] Updated weights for policy 0, policy_version 12060 (0.0010) -[2023-10-15 15:17:09,707][52866] Updated weights for policy 1, policy_version 12100 (0.0008) -[2023-10-15 15:17:10,079][52866] Updated weights for policy 1, policy_version 12110 (0.0007) -[2023-10-15 15:17:10,457][52866] Updated weights for policy 1, policy_version 12120 (0.0009) -[2023-10-15 15:17:12,881][52833] Updated weights for policy 0, policy_version 12070 (0.0008) -[2023-10-15 15:17:13,252][52833] Updated weights for policy 0, policy_version 12080 (0.0008) -[2023-10-15 15:17:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 24772608. Throughput: 0: 1806.9, 1: 1790.7. Samples: 6209228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:13,442][51532] Avg episode reward: [(0, '18.700'), (1, '19.430')] -[2023-10-15 15:17:13,627][52833] Updated weights for policy 0, policy_version 12090 (0.0008) -[2023-10-15 15:17:14,271][52866] Updated weights for policy 1, policy_version 12130 (0.0008) -[2023-10-15 15:17:14,638][52866] Updated weights for policy 1, policy_version 12140 (0.0009) -[2023-10-15 15:17:15,016][52866] Updated weights for policy 1, policy_version 12150 (0.0010) -[2023-10-15 15:17:15,387][52866] Updated weights for policy 1, policy_version 12160 (0.0008) -[2023-10-15 15:17:17,359][52833] Updated weights for policy 0, policy_version 12100 (0.0009) -[2023-10-15 15:17:17,738][52833] Updated weights for policy 0, policy_version 12110 (0.0009) -[2023-10-15 15:17:18,104][52833] Updated weights for policy 0, policy_version 12120 (0.0007) -[2023-10-15 15:17:18,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 24870912. Throughput: 0: 1777.7, 1: 1793.7. Samples: 6219404. Policy #0 lag: (min: 22.0, avg: 25.2, max: 54.0) -[2023-10-15 15:17:18,442][51532] Avg episode reward: [(0, '18.200'), (1, '19.320')] -[2023-10-15 15:17:19,027][52866] Updated weights for policy 1, policy_version 12170 (0.0008) -[2023-10-15 15:17:19,401][52866] Updated weights for policy 1, policy_version 12180 (0.0008) -[2023-10-15 15:17:19,765][52866] Updated weights for policy 1, policy_version 12190 (0.0008) -[2023-10-15 15:17:21,727][52833] Updated weights for policy 0, policy_version 12130 (0.0007) -[2023-10-15 15:17:22,104][52833] Updated weights for policy 0, policy_version 12140 (0.0007) -[2023-10-15 15:17:22,481][52833] Updated weights for policy 0, policy_version 12150 (0.0008) -[2023-10-15 15:17:22,847][52833] Updated weights for policy 0, policy_version 12160 (0.0008) -[2023-10-15 15:17:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 24936448. Throughput: 0: 1797.2, 1: 1792.7. Samples: 6241618. Policy #0 lag: (min: 22.0, avg: 25.2, max: 54.0) -[2023-10-15 15:17:23,442][51532] Avg episode reward: [(0, '19.330'), (1, '19.250')] -[2023-10-15 15:17:23,604][52866] Updated weights for policy 1, policy_version 12200 (0.0008) -[2023-10-15 15:17:23,978][52866] Updated weights for policy 1, policy_version 12210 (0.0008) -[2023-10-15 15:17:24,347][52866] Updated weights for policy 1, policy_version 12220 (0.0009) -[2023-10-15 15:17:26,490][52833] Updated weights for policy 0, policy_version 12170 (0.0007) -[2023-10-15 15:17:26,855][52833] Updated weights for policy 0, policy_version 12180 (0.0007) -[2023-10-15 15:17:27,214][52833] Updated weights for policy 0, policy_version 12190 (0.0007) -[2023-10-15 15:17:28,120][52866] Updated weights for policy 1, policy_version 12230 (0.0009) -[2023-10-15 15:17:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 25001984. Throughput: 0: 1782.1, 1: 1808.3. Samples: 6262782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:28,441][51532] Avg episode reward: [(0, '20.740'), (1, '19.510')] -[2023-10-15 15:17:28,450][52410] Saving new best policy, reward=20.740! -[2023-10-15 15:17:28,496][52866] Updated weights for policy 1, policy_version 12240 (0.0008) -[2023-10-15 15:17:28,867][52866] Updated weights for policy 1, policy_version 12250 (0.0007) -[2023-10-15 15:17:31,139][52833] Updated weights for policy 0, policy_version 12200 (0.0008) -[2023-10-15 15:17:31,511][52833] Updated weights for policy 0, policy_version 12210 (0.0008) -[2023-10-15 15:17:31,877][52833] Updated weights for policy 0, policy_version 12220 (0.0007) -[2023-10-15 15:17:32,542][52866] Updated weights for policy 1, policy_version 12260 (0.0008) -[2023-10-15 15:17:32,911][52866] Updated weights for policy 1, policy_version 12270 (0.0010) -[2023-10-15 15:17:33,274][52866] Updated weights for policy 1, policy_version 12280 (0.0008) -[2023-10-15 15:17:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 25067520. Throughput: 0: 1795.1, 1: 1788.4. Samples: 6273884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:33,442][51532] Avg episode reward: [(0, '20.780'), (1, '18.650')] -[2023-10-15 15:17:33,443][52410] Saving new best policy, reward=20.780! -[2023-10-15 15:17:35,647][52833] Updated weights for policy 0, policy_version 12230 (0.0008) -[2023-10-15 15:17:36,033][52833] Updated weights for policy 0, policy_version 12240 (0.0007) -[2023-10-15 15:17:36,401][52833] Updated weights for policy 0, policy_version 12250 (0.0009) -[2023-10-15 15:17:37,024][52866] Updated weights for policy 1, policy_version 12290 (0.0010) -[2023-10-15 15:17:37,396][52866] Updated weights for policy 1, policy_version 12300 (0.0011) -[2023-10-15 15:17:37,769][52866] Updated weights for policy 1, policy_version 12310 (0.0007) -[2023-10-15 15:17:38,130][52866] Updated weights for policy 1, policy_version 12320 (0.0008) -[2023-10-15 15:17:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 25165824. Throughput: 0: 1769.7, 1: 1807.4. Samples: 6294656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:38,441][51532] Avg episode reward: [(0, '19.360'), (1, '19.060')] -[2023-10-15 15:17:40,039][52833] Updated weights for policy 0, policy_version 12260 (0.0008) -[2023-10-15 15:17:40,413][52833] Updated weights for policy 0, policy_version 12270 (0.0009) -[2023-10-15 15:17:40,784][52833] Updated weights for policy 0, policy_version 12280 (0.0008) -[2023-10-15 15:17:41,967][52866] Updated weights for policy 1, policy_version 12330 (0.0008) -[2023-10-15 15:17:42,341][52866] Updated weights for policy 1, policy_version 12340 (0.0008) -[2023-10-15 15:17:42,710][52866] Updated weights for policy 1, policy_version 12350 (0.0008) -[2023-10-15 15:17:43,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 25231360. Throughput: 0: 1771.7, 1: 1787.5. Samples: 6315588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:43,442][51532] Avg episode reward: [(0, '17.480'), (1, '18.110')] -[2023-10-15 15:17:43,457][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000012352_12648448.pth... -[2023-10-15 15:17:43,457][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000012288_12582912.pth... -[2023-10-15 15:17:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000010624_10878976.pth -[2023-10-15 15:17:43,499][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000010656_10911744.pth -[2023-10-15 15:17:44,554][52833] Updated weights for policy 0, policy_version 12290 (0.0009) -[2023-10-15 15:17:44,924][52833] Updated weights for policy 0, policy_version 12300 (0.0010) -[2023-10-15 15:17:45,292][52833] Updated weights for policy 0, policy_version 12310 (0.0011) -[2023-10-15 15:17:45,662][52833] Updated weights for policy 0, policy_version 12320 (0.0011) -[2023-10-15 15:17:46,422][52866] Updated weights for policy 1, policy_version 12360 (0.0010) -[2023-10-15 15:17:46,790][52866] Updated weights for policy 1, policy_version 12370 (0.0007) -[2023-10-15 15:17:47,149][52866] Updated weights for policy 1, policy_version 12380 (0.0009) -[2023-10-15 15:17:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 25296896. Throughput: 0: 1773.6, 1: 1799.6. Samples: 6326754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:48,444][51532] Avg episode reward: [(0, '17.270'), (1, '18.550')] -[2023-10-15 15:17:49,534][52833] Updated weights for policy 0, policy_version 12330 (0.0008) -[2023-10-15 15:17:49,913][52833] Updated weights for policy 0, policy_version 12340 (0.0007) -[2023-10-15 15:17:50,293][52833] Updated weights for policy 0, policy_version 12350 (0.0011) -[2023-10-15 15:17:51,172][52866] Updated weights for policy 1, policy_version 12390 (0.0009) -[2023-10-15 15:17:51,535][52866] Updated weights for policy 1, policy_version 12400 (0.0010) -[2023-10-15 15:17:51,908][52866] Updated weights for policy 1, policy_version 12410 (0.0010) -[2023-10-15 15:17:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 25362432. Throughput: 0: 1780.2, 1: 1786.1. Samples: 6347980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:17:53,442][51532] Avg episode reward: [(0, '17.040'), (1, '17.930')] -[2023-10-15 15:17:53,959][52833] Updated weights for policy 0, policy_version 12360 (0.0009) -[2023-10-15 15:17:54,329][52833] Updated weights for policy 0, policy_version 12370 (0.0007) -[2023-10-15 15:17:54,702][52833] Updated weights for policy 0, policy_version 12380 (0.0008) -[2023-10-15 15:17:55,609][52866] Updated weights for policy 1, policy_version 12420 (0.0008) -[2023-10-15 15:17:55,976][52866] Updated weights for policy 1, policy_version 12430 (0.0007) -[2023-10-15 15:17:56,342][52866] Updated weights for policy 1, policy_version 12440 (0.0011) -[2023-10-15 15:17:58,360][52833] Updated weights for policy 0, policy_version 12390 (0.0007) -[2023-10-15 15:17:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 25427968. Throughput: 0: 1795.0, 1: 1782.3. Samples: 6370204. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) -[2023-10-15 15:17:58,442][51532] Avg episode reward: [(0, '18.450'), (1, '18.580')] -[2023-10-15 15:17:58,724][52833] Updated weights for policy 0, policy_version 12400 (0.0008) -[2023-10-15 15:17:59,096][52833] Updated weights for policy 0, policy_version 12410 (0.0011) -[2023-10-15 15:18:00,106][52866] Updated weights for policy 1, policy_version 12450 (0.0010) -[2023-10-15 15:18:00,467][52866] Updated weights for policy 1, policy_version 12460 (0.0007) -[2023-10-15 15:18:00,841][52866] Updated weights for policy 1, policy_version 12470 (0.0009) -[2023-10-15 15:18:01,210][52866] Updated weights for policy 1, policy_version 12480 (0.0009) -[2023-10-15 15:18:03,008][52833] Updated weights for policy 0, policy_version 12420 (0.0009) -[2023-10-15 15:18:03,374][52833] Updated weights for policy 0, policy_version 12430 (0.0007) -[2023-10-15 15:18:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 25493504. Throughput: 0: 1787.6, 1: 1792.7. Samples: 6380516. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) -[2023-10-15 15:18:03,441][51532] Avg episode reward: [(0, '18.890'), (1, '18.890')] -[2023-10-15 15:18:03,750][52833] Updated weights for policy 0, policy_version 12440 (0.0007) -[2023-10-15 15:18:04,929][52866] Updated weights for policy 1, policy_version 12490 (0.0008) -[2023-10-15 15:18:05,294][52866] Updated weights for policy 1, policy_version 12500 (0.0007) -[2023-10-15 15:18:05,661][52866] Updated weights for policy 1, policy_version 12510 (0.0007) -[2023-10-15 15:18:07,375][52833] Updated weights for policy 0, policy_version 12450 (0.0009) -[2023-10-15 15:18:07,745][52833] Updated weights for policy 0, policy_version 12460 (0.0007) -[2023-10-15 15:18:08,113][52833] Updated weights for policy 0, policy_version 12470 (0.0007) -[2023-10-15 15:18:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 25559040. Throughput: 0: 1793.7, 1: 1782.8. Samples: 6402560. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) -[2023-10-15 15:18:08,442][51532] Avg episode reward: [(0, '17.610'), (1, '20.480')] -[2023-10-15 15:18:08,486][52833] Updated weights for policy 0, policy_version 12480 (0.0007) -[2023-10-15 15:18:09,309][52866] Updated weights for policy 1, policy_version 12520 (0.0008) -[2023-10-15 15:18:09,675][52866] Updated weights for policy 1, policy_version 12530 (0.0008) -[2023-10-15 15:18:10,050][52866] Updated weights for policy 1, policy_version 12540 (0.0008) -[2023-10-15 15:18:12,262][52833] Updated weights for policy 0, policy_version 12490 (0.0008) -[2023-10-15 15:18:12,628][52833] Updated weights for policy 0, policy_version 12500 (0.0007) -[2023-10-15 15:18:13,007][52833] Updated weights for policy 0, policy_version 12510 (0.0008) -[2023-10-15 15:18:13,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 25657344. Throughput: 0: 1794.3, 1: 1789.7. Samples: 6424064. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:18:13,442][51532] Avg episode reward: [(0, '17.110'), (1, '20.670')] -[2023-10-15 15:18:13,791][52866] Updated weights for policy 1, policy_version 12550 (0.0009) -[2023-10-15 15:18:14,163][52866] Updated weights for policy 1, policy_version 12560 (0.0009) -[2023-10-15 15:18:14,542][52866] Updated weights for policy 1, policy_version 12570 (0.0008) -[2023-10-15 15:18:16,797][52833] Updated weights for policy 0, policy_version 12520 (0.0009) -[2023-10-15 15:18:17,170][52833] Updated weights for policy 0, policy_version 12530 (0.0009) -[2023-10-15 15:18:17,540][52833] Updated weights for policy 0, policy_version 12540 (0.0008) -[2023-10-15 15:18:18,332][52866] Updated weights for policy 1, policy_version 12580 (0.0009) -[2023-10-15 15:18:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 25722880. Throughput: 0: 1793.7, 1: 1785.2. Samples: 6434938. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:18:18,442][51532] Avg episode reward: [(0, '17.070'), (1, '20.890')] -[2023-10-15 15:18:18,707][52866] Updated weights for policy 1, policy_version 12590 (0.0008) -[2023-10-15 15:18:19,076][52866] Updated weights for policy 1, policy_version 12600 (0.0008) -[2023-10-15 15:18:21,250][52833] Updated weights for policy 0, policy_version 12550 (0.0008) -[2023-10-15 15:18:21,631][52833] Updated weights for policy 0, policy_version 12560 (0.0009) -[2023-10-15 15:18:21,990][52833] Updated weights for policy 0, policy_version 12570 (0.0007) -[2023-10-15 15:18:23,032][52866] Updated weights for policy 1, policy_version 12610 (0.0007) -[2023-10-15 15:18:23,402][52866] Updated weights for policy 1, policy_version 12620 (0.0008) -[2023-10-15 15:18:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 25788416. Throughput: 0: 1808.5, 1: 1782.3. Samples: 6456242. Policy #0 lag: (min: 12.0, avg: 17.9, max: 44.0) -[2023-10-15 15:18:23,441][51532] Avg episode reward: [(0, '16.560'), (1, '20.970')] -[2023-10-15 15:18:23,765][52866] Updated weights for policy 1, policy_version 12630 (0.0009) -[2023-10-15 15:18:24,130][52866] Updated weights for policy 1, policy_version 12640 (0.0010) -[2023-10-15 15:18:25,818][52833] Updated weights for policy 0, policy_version 12580 (0.0007) -[2023-10-15 15:18:26,199][52833] Updated weights for policy 0, policy_version 12590 (0.0008) -[2023-10-15 15:18:26,562][52833] Updated weights for policy 0, policy_version 12600 (0.0007) -[2023-10-15 15:18:27,823][52866] Updated weights for policy 1, policy_version 12650 (0.0010) -[2023-10-15 15:18:28,191][52866] Updated weights for policy 1, policy_version 12660 (0.0010) -[2023-10-15 15:18:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 25853952. Throughput: 0: 1798.2, 1: 1800.2. Samples: 6477514. Policy #0 lag: (min: 12.0, avg: 17.9, max: 44.0) -[2023-10-15 15:18:28,441][51532] Avg episode reward: [(0, '18.880'), (1, '21.470')] -[2023-10-15 15:18:28,569][52866] Updated weights for policy 1, policy_version 12670 (0.0008) -[2023-10-15 15:18:30,214][52833] Updated weights for policy 0, policy_version 12610 (0.0008) -[2023-10-15 15:18:30,588][52833] Updated weights for policy 0, policy_version 12620 (0.0008) -[2023-10-15 15:18:30,953][52833] Updated weights for policy 0, policy_version 12630 (0.0009) -[2023-10-15 15:18:31,326][52833] Updated weights for policy 0, policy_version 12640 (0.0008) -[2023-10-15 15:18:32,244][52866] Updated weights for policy 1, policy_version 12680 (0.0008) -[2023-10-15 15:18:32,614][52866] Updated weights for policy 1, policy_version 12690 (0.0008) -[2023-10-15 15:18:32,981][52866] Updated weights for policy 1, policy_version 12700 (0.0009) -[2023-10-15 15:18:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 25952256. Throughput: 0: 1817.8, 1: 1783.3. Samples: 6488806. Policy #0 lag: (min: 12.0, avg: 17.9, max: 44.0) -[2023-10-15 15:18:33,442][51532] Avg episode reward: [(0, '19.120'), (1, '19.910')] -[2023-10-15 15:18:34,920][52833] Updated weights for policy 0, policy_version 12650 (0.0011) -[2023-10-15 15:18:35,288][52833] Updated weights for policy 0, policy_version 12660 (0.0011) -[2023-10-15 15:18:35,663][52833] Updated weights for policy 0, policy_version 12670 (0.0008) -[2023-10-15 15:18:36,675][52866] Updated weights for policy 1, policy_version 12710 (0.0008) -[2023-10-15 15:18:37,046][52866] Updated weights for policy 1, policy_version 12720 (0.0011) -[2023-10-15 15:18:37,418][52866] Updated weights for policy 1, policy_version 12730 (0.0010) -[2023-10-15 15:18:38,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 26017792. Throughput: 0: 1800.7, 1: 1802.4. Samples: 6510118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:18:38,441][51532] Avg episode reward: [(0, '21.370'), (1, '19.340')] -[2023-10-15 15:18:38,442][52410] Saving new best policy, reward=21.370! -[2023-10-15 15:18:39,571][52833] Updated weights for policy 0, policy_version 12680 (0.0008) -[2023-10-15 15:18:39,930][52833] Updated weights for policy 0, policy_version 12690 (0.0008) -[2023-10-15 15:18:40,299][52833] Updated weights for policy 0, policy_version 12700 (0.0009) -[2023-10-15 15:18:41,089][52866] Updated weights for policy 1, policy_version 12740 (0.0011) -[2023-10-15 15:18:41,461][52866] Updated weights for policy 1, policy_version 12750 (0.0009) -[2023-10-15 15:18:41,822][52866] Updated weights for policy 1, policy_version 12760 (0.0010) -[2023-10-15 15:18:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14329.1). Total num frames: 26083328. Throughput: 0: 1799.8, 1: 1795.4. Samples: 6531986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:18:43,441][51532] Avg episode reward: [(0, '19.780'), (1, '19.630')] -[2023-10-15 15:18:44,071][52833] Updated weights for policy 0, policy_version 12710 (0.0008) -[2023-10-15 15:18:44,456][52833] Updated weights for policy 0, policy_version 12720 (0.0009) -[2023-10-15 15:18:44,829][52833] Updated weights for policy 0, policy_version 12730 (0.0008) -[2023-10-15 15:18:45,551][52866] Updated weights for policy 1, policy_version 12770 (0.0008) -[2023-10-15 15:18:45,930][52866] Updated weights for policy 1, policy_version 12780 (0.0007) -[2023-10-15 15:18:46,311][52866] Updated weights for policy 1, policy_version 12790 (0.0009) -[2023-10-15 15:18:46,671][52866] Updated weights for policy 1, policy_version 12800 (0.0010) -[2023-10-15 15:18:48,359][52833] Updated weights for policy 0, policy_version 12740 (0.0011) -[2023-10-15 15:18:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 26148864. Throughput: 0: 1798.0, 1: 1803.9. Samples: 6542598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:18:48,442][51532] Avg episode reward: [(0, '19.030'), (1, '17.430')] -[2023-10-15 15:18:48,740][52833] Updated weights for policy 0, policy_version 12750 (0.0009) -[2023-10-15 15:18:49,110][52833] Updated weights for policy 0, policy_version 12760 (0.0008) -[2023-10-15 15:18:50,510][52866] Updated weights for policy 1, policy_version 12810 (0.0008) -[2023-10-15 15:18:50,880][52866] Updated weights for policy 1, policy_version 12820 (0.0008) -[2023-10-15 15:18:51,240][52866] Updated weights for policy 1, policy_version 12830 (0.0009) -[2023-10-15 15:18:53,045][52833] Updated weights for policy 0, policy_version 12770 (0.0008) -[2023-10-15 15:18:53,414][52833] Updated weights for policy 0, policy_version 12780 (0.0007) -[2023-10-15 15:18:53,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 26214400. Throughput: 0: 1796.3, 1: 1789.6. Samples: 6563926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-15 15:18:53,442][51532] Avg episode reward: [(0, '20.230'), (1, '17.620')] -[2023-10-15 15:18:53,785][52833] Updated weights for policy 0, policy_version 12790 (0.0010) -[2023-10-15 15:18:54,154][52833] Updated weights for policy 0, policy_version 12800 (0.0009) -[2023-10-15 15:18:55,043][52866] Updated weights for policy 1, policy_version 12840 (0.0009) -[2023-10-15 15:18:55,412][52866] Updated weights for policy 1, policy_version 12850 (0.0008) -[2023-10-15 15:18:55,785][52866] Updated weights for policy 1, policy_version 12860 (0.0007) -[2023-10-15 15:18:57,973][52833] Updated weights for policy 0, policy_version 12810 (0.0009) -[2023-10-15 15:18:58,335][52833] Updated weights for policy 0, policy_version 12820 (0.0009) -[2023-10-15 15:18:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 26279936. Throughput: 0: 1811.2, 1: 1787.5. Samples: 6586004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-15 15:18:58,441][51532] Avg episode reward: [(0, '19.450'), (1, '19.010')] -[2023-10-15 15:18:58,713][52833] Updated weights for policy 0, policy_version 12830 (0.0011) -[2023-10-15 15:18:59,554][52866] Updated weights for policy 1, policy_version 12870 (0.0007) -[2023-10-15 15:18:59,925][52866] Updated weights for policy 1, policy_version 12880 (0.0010) -[2023-10-15 15:19:00,300][52866] Updated weights for policy 1, policy_version 12890 (0.0009) -[2023-10-15 15:19:02,505][52833] Updated weights for policy 0, policy_version 12840 (0.0009) -[2023-10-15 15:19:02,880][52833] Updated weights for policy 0, policy_version 12850 (0.0007) -[2023-10-15 15:19:03,262][52833] Updated weights for policy 0, policy_version 12860 (0.0008) -[2023-10-15 15:19:03,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 26378240. Throughput: 0: 1791.5, 1: 1786.5. Samples: 6595948. Policy #0 lag: (min: 24.0, avg: 50.8, max: 56.0) -[2023-10-15 15:19:03,442][51532] Avg episode reward: [(0, '18.960'), (1, '19.410')] -[2023-10-15 15:19:03,879][52866] Updated weights for policy 1, policy_version 12900 (0.0009) -[2023-10-15 15:19:04,250][52866] Updated weights for policy 1, policy_version 12910 (0.0008) -[2023-10-15 15:19:04,611][52866] Updated weights for policy 1, policy_version 12920 (0.0007) -[2023-10-15 15:19:07,140][52833] Updated weights for policy 0, policy_version 12870 (0.0007) -[2023-10-15 15:19:07,520][52833] Updated weights for policy 0, policy_version 12880 (0.0007) -[2023-10-15 15:19:07,895][52833] Updated weights for policy 0, policy_version 12890 (0.0007) -[2023-10-15 15:19:08,436][52866] Updated weights for policy 1, policy_version 12930 (0.0010) -[2023-10-15 15:19:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 26443776. Throughput: 0: 1807.6, 1: 1790.8. Samples: 6618168. Policy #0 lag: (min: 24.0, avg: 50.8, max: 56.0) -[2023-10-15 15:19:08,441][51532] Avg episode reward: [(0, '18.940'), (1, '19.020')] -[2023-10-15 15:19:08,807][52866] Updated weights for policy 1, policy_version 12940 (0.0007) -[2023-10-15 15:19:09,171][52866] Updated weights for policy 1, policy_version 12950 (0.0008) -[2023-10-15 15:19:09,535][52866] Updated weights for policy 1, policy_version 12960 (0.0007) -[2023-10-15 15:19:11,495][52833] Updated weights for policy 0, policy_version 12900 (0.0010) -[2023-10-15 15:19:11,859][52833] Updated weights for policy 0, policy_version 12910 (0.0007) -[2023-10-15 15:19:12,214][52833] Updated weights for policy 0, policy_version 12920 (0.0007) -[2023-10-15 15:19:13,285][52866] Updated weights for policy 1, policy_version 12970 (0.0011) -[2023-10-15 15:19:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 26509312. Throughput: 0: 1781.3, 1: 1809.0. Samples: 6639078. Policy #0 lag: (min: 24.0, avg: 50.8, max: 56.0) -[2023-10-15 15:19:13,442][51532] Avg episode reward: [(0, '17.210'), (1, '20.200')] -[2023-10-15 15:19:13,640][52866] Updated weights for policy 1, policy_version 12980 (0.0009) -[2023-10-15 15:19:14,005][52866] Updated weights for policy 1, policy_version 12990 (0.0010) -[2023-10-15 15:19:15,932][52833] Updated weights for policy 0, policy_version 12930 (0.0007) -[2023-10-15 15:19:16,298][52833] Updated weights for policy 0, policy_version 12940 (0.0009) -[2023-10-15 15:19:16,671][52833] Updated weights for policy 0, policy_version 12950 (0.0010) -[2023-10-15 15:19:17,036][52833] Updated weights for policy 0, policy_version 12960 (0.0008) -[2023-10-15 15:19:17,894][52866] Updated weights for policy 1, policy_version 13000 (0.0007) -[2023-10-15 15:19:18,258][52866] Updated weights for policy 1, policy_version 13010 (0.0009) -[2023-10-15 15:19:18,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 26574848. Throughput: 0: 1802.9, 1: 1789.7. Samples: 6650476. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) -[2023-10-15 15:19:18,442][51532] Avg episode reward: [(0, '17.780'), (1, '19.630')] -[2023-10-15 15:19:18,622][52866] Updated weights for policy 1, policy_version 13020 (0.0010) -[2023-10-15 15:19:20,986][52833] Updated weights for policy 0, policy_version 12970 (0.0007) -[2023-10-15 15:19:21,360][52833] Updated weights for policy 0, policy_version 12980 (0.0010) -[2023-10-15 15:19:21,734][52833] Updated weights for policy 0, policy_version 12990 (0.0009) -[2023-10-15 15:19:22,517][52866] Updated weights for policy 1, policy_version 13030 (0.0009) -[2023-10-15 15:19:22,868][52866] Updated weights for policy 1, policy_version 13040 (0.0011) -[2023-10-15 15:19:23,237][52866] Updated weights for policy 1, policy_version 13050 (0.0008) -[2023-10-15 15:19:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 26640384. Throughput: 0: 1779.9, 1: 1799.0. Samples: 6671166. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) -[2023-10-15 15:19:23,442][51532] Avg episode reward: [(0, '18.260'), (1, '19.560')] -[2023-10-15 15:19:25,417][52833] Updated weights for policy 0, policy_version 13000 (0.0008) -[2023-10-15 15:19:25,792][52833] Updated weights for policy 0, policy_version 13010 (0.0007) -[2023-10-15 15:19:26,155][52833] Updated weights for policy 0, policy_version 13020 (0.0008) -[2023-10-15 15:19:27,080][52866] Updated weights for policy 1, policy_version 13060 (0.0008) -[2023-10-15 15:19:27,444][52866] Updated weights for policy 1, policy_version 13070 (0.0008) -[2023-10-15 15:19:27,812][52866] Updated weights for policy 1, policy_version 13080 (0.0007) -[2023-10-15 15:19:28,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 26738688. Throughput: 0: 1781.4, 1: 1785.9. Samples: 6692516. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) -[2023-10-15 15:19:28,442][51532] Avg episode reward: [(0, '18.780'), (1, '19.830')] -[2023-10-15 15:19:29,774][52833] Updated weights for policy 0, policy_version 13030 (0.0007) -[2023-10-15 15:19:30,139][52833] Updated weights for policy 0, policy_version 13040 (0.0008) -[2023-10-15 15:19:30,516][52833] Updated weights for policy 0, policy_version 13050 (0.0007) -[2023-10-15 15:19:31,646][52866] Updated weights for policy 1, policy_version 13090 (0.0009) -[2023-10-15 15:19:32,016][52866] Updated weights for policy 1, policy_version 13100 (0.0008) -[2023-10-15 15:19:32,375][52866] Updated weights for policy 1, policy_version 13110 (0.0007) -[2023-10-15 15:19:32,751][52866] Updated weights for policy 1, policy_version 13120 (0.0008) -[2023-10-15 15:19:33,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 26804224. Throughput: 0: 1783.3, 1: 1791.9. Samples: 6703482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:19:33,441][51532] Avg episode reward: [(0, '19.070'), (1, '19.200')] -[2023-10-15 15:19:34,231][52833] Updated weights for policy 0, policy_version 13060 (0.0008) -[2023-10-15 15:19:34,602][52833] Updated weights for policy 0, policy_version 13070 (0.0008) -[2023-10-15 15:19:34,977][52833] Updated weights for policy 0, policy_version 13080 (0.0007) -[2023-10-15 15:19:36,449][52866] Updated weights for policy 1, policy_version 13130 (0.0010) -[2023-10-15 15:19:36,828][52866] Updated weights for policy 1, policy_version 13140 (0.0007) -[2023-10-15 15:19:37,195][52866] Updated weights for policy 1, policy_version 13150 (0.0008) -[2023-10-15 15:19:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 26869760. Throughput: 0: 1786.7, 1: 1790.8. Samples: 6724912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:19:38,442][51532] Avg episode reward: [(0, '19.610'), (1, '18.400')] -[2023-10-15 15:19:38,734][52833] Updated weights for policy 0, policy_version 13090 (0.0007) -[2023-10-15 15:19:39,104][52833] Updated weights for policy 0, policy_version 13100 (0.0011) -[2023-10-15 15:19:39,473][52833] Updated weights for policy 0, policy_version 13110 (0.0010) -[2023-10-15 15:19:39,834][52833] Updated weights for policy 0, policy_version 13120 (0.0010) -[2023-10-15 15:19:40,836][52866] Updated weights for policy 1, policy_version 13160 (0.0009) -[2023-10-15 15:19:41,199][52866] Updated weights for policy 1, policy_version 13170 (0.0008) -[2023-10-15 15:19:41,563][52866] Updated weights for policy 1, policy_version 13180 (0.0007) -[2023-10-15 15:19:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 26935296. Throughput: 0: 1795.3, 1: 1784.5. Samples: 6747096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:19:43,441][51532] Avg episode reward: [(0, '19.470'), (1, '18.820')] -[2023-10-15 15:19:43,448][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000013184_13500416.pth... -[2023-10-15 15:19:43,477][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000011520_11796480.pth -[2023-10-15 15:19:43,628][52833] Updated weights for policy 0, policy_version 13130 (0.0008) -[2023-10-15 15:19:43,991][52833] Updated weights for policy 0, policy_version 13140 (0.0007) -[2023-10-15 15:19:44,374][52833] Updated weights for policy 0, policy_version 13150 (0.0009) -[2023-10-15 15:19:44,445][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000013152_13467648.pth... -[2023-10-15 15:19:44,484][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000011456_11730944.pth -[2023-10-15 15:19:45,519][52866] Updated weights for policy 1, policy_version 13190 (0.0009) -[2023-10-15 15:19:45,900][52866] Updated weights for policy 1, policy_version 13200 (0.0009) -[2023-10-15 15:19:46,271][52866] Updated weights for policy 1, policy_version 13210 (0.0010) -[2023-10-15 15:19:48,234][52833] Updated weights for policy 0, policy_version 13160 (0.0011) -[2023-10-15 15:19:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 27000832. Throughput: 0: 1787.7, 1: 1798.1. Samples: 6757310. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-15 15:19:48,441][51532] Avg episode reward: [(0, '19.950'), (1, '19.490')] -[2023-10-15 15:19:48,612][52833] Updated weights for policy 0, policy_version 13170 (0.0010) -[2023-10-15 15:19:48,984][52833] Updated weights for policy 0, policy_version 13180 (0.0007) -[2023-10-15 15:19:50,050][52866] Updated weights for policy 1, policy_version 13220 (0.0008) -[2023-10-15 15:19:50,417][52866] Updated weights for policy 1, policy_version 13230 (0.0007) -[2023-10-15 15:19:50,783][52866] Updated weights for policy 1, policy_version 13240 (0.0008) -[2023-10-15 15:19:52,700][52833] Updated weights for policy 0, policy_version 13190 (0.0008) -[2023-10-15 15:19:53,079][52833] Updated weights for policy 0, policy_version 13200 (0.0007) -[2023-10-15 15:19:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 27066368. Throughput: 0: 1793.1, 1: 1777.3. Samples: 6778836. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-15 15:19:53,442][51532] Avg episode reward: [(0, '20.730'), (1, '18.750')] -[2023-10-15 15:19:53,446][52833] Updated weights for policy 0, policy_version 13210 (0.0008) -[2023-10-15 15:19:54,516][52866] Updated weights for policy 1, policy_version 13250 (0.0009) -[2023-10-15 15:19:54,884][52866] Updated weights for policy 1, policy_version 13260 (0.0007) -[2023-10-15 15:19:55,249][52866] Updated weights for policy 1, policy_version 13270 (0.0007) -[2023-10-15 15:19:55,620][52866] Updated weights for policy 1, policy_version 13280 (0.0007) -[2023-10-15 15:19:57,168][52833] Updated weights for policy 0, policy_version 13220 (0.0009) -[2023-10-15 15:19:57,546][52833] Updated weights for policy 0, policy_version 13230 (0.0008) -[2023-10-15 15:19:57,913][52833] Updated weights for policy 0, policy_version 13240 (0.0007) -[2023-10-15 15:19:58,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 27164672. Throughput: 0: 1803.2, 1: 1777.9. Samples: 6800224. Policy #0 lag: (min: 22.0, avg: 39.1, max: 40.0) -[2023-10-15 15:19:58,442][51532] Avg episode reward: [(0, '21.620'), (1, '19.200')] -[2023-10-15 15:19:58,453][52410] Saving new best policy, reward=21.620! -[2023-10-15 15:19:59,390][52866] Updated weights for policy 1, policy_version 13290 (0.0009) -[2023-10-15 15:19:59,761][52866] Updated weights for policy 1, policy_version 13300 (0.0008) -[2023-10-15 15:20:00,142][52866] Updated weights for policy 1, policy_version 13310 (0.0010) -[2023-10-15 15:20:01,540][52833] Updated weights for policy 0, policy_version 13250 (0.0009) -[2023-10-15 15:20:01,904][52833] Updated weights for policy 0, policy_version 13260 (0.0009) -[2023-10-15 15:20:02,282][52833] Updated weights for policy 0, policy_version 13270 (0.0009) -[2023-10-15 15:20:02,644][52833] Updated weights for policy 0, policy_version 13280 (0.0008) -[2023-10-15 15:20:03,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 27230208. Throughput: 0: 1788.2, 1: 1776.5. Samples: 6810886. Policy #0 lag: (min: 22.0, avg: 39.1, max: 40.0) -[2023-10-15 15:20:03,441][51532] Avg episode reward: [(0, '21.810'), (1, '19.420')] -[2023-10-15 15:20:03,442][52410] Saving new best policy, reward=21.810! -[2023-10-15 15:20:04,065][52866] Updated weights for policy 1, policy_version 13320 (0.0010) -[2023-10-15 15:20:04,435][52866] Updated weights for policy 1, policy_version 13330 (0.0008) -[2023-10-15 15:20:04,812][52866] Updated weights for policy 1, policy_version 13340 (0.0007) -[2023-10-15 15:20:06,426][52833] Updated weights for policy 0, policy_version 13290 (0.0009) -[2023-10-15 15:20:06,796][52833] Updated weights for policy 0, policy_version 13300 (0.0009) -[2023-10-15 15:20:07,159][52833] Updated weights for policy 0, policy_version 13310 (0.0010) -[2023-10-15 15:20:08,426][52866] Updated weights for policy 1, policy_version 13350 (0.0007) -[2023-10-15 15:20:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 27295744. Throughput: 0: 1804.7, 1: 1774.9. Samples: 6832252. Policy #0 lag: (min: 22.0, avg: 39.1, max: 40.0) -[2023-10-15 15:20:08,442][51532] Avg episode reward: [(0, '21.820'), (1, '19.220')] -[2023-10-15 15:20:08,443][52410] Saving new best policy, reward=21.820! -[2023-10-15 15:20:08,797][52866] Updated weights for policy 1, policy_version 13360 (0.0009) -[2023-10-15 15:20:09,169][52866] Updated weights for policy 1, policy_version 13370 (0.0007) -[2023-10-15 15:20:10,897][52833] Updated weights for policy 0, policy_version 13320 (0.0008) -[2023-10-15 15:20:11,267][52833] Updated weights for policy 0, policy_version 13330 (0.0009) -[2023-10-15 15:20:11,633][52833] Updated weights for policy 0, policy_version 13340 (0.0010) -[2023-10-15 15:20:12,867][52866] Updated weights for policy 1, policy_version 13380 (0.0007) -[2023-10-15 15:20:13,236][52866] Updated weights for policy 1, policy_version 13390 (0.0011) -[2023-10-15 15:20:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 27361280. Throughput: 0: 1790.1, 1: 1799.7. Samples: 6854056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:20:13,441][51532] Avg episode reward: [(0, '20.620'), (1, '19.270')] -[2023-10-15 15:20:13,601][52866] Updated weights for policy 1, policy_version 13400 (0.0009) -[2023-10-15 15:20:15,473][52833] Updated weights for policy 0, policy_version 13350 (0.0008) -[2023-10-15 15:20:15,838][52833] Updated weights for policy 0, policy_version 13360 (0.0007) -[2023-10-15 15:20:16,204][52833] Updated weights for policy 0, policy_version 13370 (0.0008) -[2023-10-15 15:20:17,168][52866] Updated weights for policy 1, policy_version 13410 (0.0010) -[2023-10-15 15:20:17,534][52866] Updated weights for policy 1, policy_version 13420 (0.0010) -[2023-10-15 15:20:17,901][52866] Updated weights for policy 1, policy_version 13430 (0.0007) -[2023-10-15 15:20:18,275][52866] Updated weights for policy 1, policy_version 13440 (0.0007) -[2023-10-15 15:20:18,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 27459584. Throughput: 0: 1803.2, 1: 1782.4. Samples: 6864836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:20:18,441][51532] Avg episode reward: [(0, '20.490'), (1, '18.800')] -[2023-10-15 15:20:20,085][52833] Updated weights for policy 0, policy_version 13380 (0.0008) -[2023-10-15 15:20:20,456][52833] Updated weights for policy 0, policy_version 13390 (0.0007) -[2023-10-15 15:20:20,837][52833] Updated weights for policy 0, policy_version 13400 (0.0008) -[2023-10-15 15:20:21,989][52866] Updated weights for policy 1, policy_version 13450 (0.0007) -[2023-10-15 15:20:22,355][52866] Updated weights for policy 1, policy_version 13460 (0.0007) -[2023-10-15 15:20:22,729][52866] Updated weights for policy 1, policy_version 13470 (0.0007) -[2023-10-15 15:20:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 27525120. Throughput: 0: 1785.2, 1: 1799.0. Samples: 6886204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:20:23,441][51532] Avg episode reward: [(0, '19.450'), (1, '18.870')] -[2023-10-15 15:20:24,657][52833] Updated weights for policy 0, policy_version 13410 (0.0010) -[2023-10-15 15:20:25,031][52833] Updated weights for policy 0, policy_version 13420 (0.0009) -[2023-10-15 15:20:25,398][52833] Updated weights for policy 0, policy_version 13430 (0.0010) -[2023-10-15 15:20:25,773][52833] Updated weights for policy 0, policy_version 13440 (0.0009) -[2023-10-15 15:20:26,747][52866] Updated weights for policy 1, policy_version 13480 (0.0007) -[2023-10-15 15:20:27,116][52866] Updated weights for policy 1, policy_version 13490 (0.0007) -[2023-10-15 15:20:27,490][52866] Updated weights for policy 1, policy_version 13500 (0.0008) -[2023-10-15 15:20:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 27590656. Throughput: 0: 1786.4, 1: 1780.9. Samples: 6907624. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 15:20:28,442][51532] Avg episode reward: [(0, '19.340'), (1, '20.980')] -[2023-10-15 15:20:29,420][52833] Updated weights for policy 0, policy_version 13450 (0.0008) -[2023-10-15 15:20:29,790][52833] Updated weights for policy 0, policy_version 13460 (0.0009) -[2023-10-15 15:20:30,155][52833] Updated weights for policy 0, policy_version 13470 (0.0010) -[2023-10-15 15:20:31,261][52866] Updated weights for policy 1, policy_version 13510 (0.0008) -[2023-10-15 15:20:31,641][52866] Updated weights for policy 1, policy_version 13520 (0.0010) -[2023-10-15 15:20:32,012][52866] Updated weights for policy 1, policy_version 13530 (0.0009) -[2023-10-15 15:20:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 27656192. Throughput: 0: 1784.6, 1: 1801.7. Samples: 6918696. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 15:20:33,442][51532] Avg episode reward: [(0, '18.230'), (1, '20.220')] -[2023-10-15 15:20:33,866][52833] Updated weights for policy 0, policy_version 13480 (0.0010) -[2023-10-15 15:20:34,230][52833] Updated weights for policy 0, policy_version 13490 (0.0007) -[2023-10-15 15:20:34,601][52833] Updated weights for policy 0, policy_version 13500 (0.0008) -[2023-10-15 15:20:35,704][52866] Updated weights for policy 1, policy_version 13540 (0.0007) -[2023-10-15 15:20:36,077][52866] Updated weights for policy 1, policy_version 13550 (0.0007) -[2023-10-15 15:20:36,438][52866] Updated weights for policy 1, policy_version 13560 (0.0009) -[2023-10-15 15:20:38,352][52833] Updated weights for policy 0, policy_version 13510 (0.0008) -[2023-10-15 15:20:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 27721728. Throughput: 0: 1790.5, 1: 1793.1. Samples: 6940096. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 15:20:38,442][51532] Avg episode reward: [(0, '18.810'), (1, '20.390')] -[2023-10-15 15:20:38,719][52833] Updated weights for policy 0, policy_version 13520 (0.0008) -[2023-10-15 15:20:39,088][52833] Updated weights for policy 0, policy_version 13530 (0.0007) -[2023-10-15 15:20:40,037][52866] Updated weights for policy 1, policy_version 13570 (0.0007) -[2023-10-15 15:20:40,417][52866] Updated weights for policy 1, policy_version 13580 (0.0008) -[2023-10-15 15:20:40,787][52866] Updated weights for policy 1, policy_version 13590 (0.0008) -[2023-10-15 15:20:41,158][52866] Updated weights for policy 1, policy_version 13600 (0.0009) -[2023-10-15 15:20:42,741][52833] Updated weights for policy 0, policy_version 13540 (0.0008) -[2023-10-15 15:20:43,114][52833] Updated weights for policy 0, policy_version 13550 (0.0008) -[2023-10-15 15:20:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 27787264. Throughput: 0: 1808.6, 1: 1796.8. Samples: 6962468. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-15 15:20:43,441][51532] Avg episode reward: [(0, '19.030'), (1, '21.720')] -[2023-10-15 15:20:43,450][52518] Saving new best policy, reward=21.720! -[2023-10-15 15:20:43,485][52833] Updated weights for policy 0, policy_version 13560 (0.0011) -[2023-10-15 15:20:44,781][52866] Updated weights for policy 1, policy_version 13610 (0.0009) -[2023-10-15 15:20:45,157][52866] Updated weights for policy 1, policy_version 13620 (0.0010) -[2023-10-15 15:20:45,530][52866] Updated weights for policy 1, policy_version 13630 (0.0011) -[2023-10-15 15:20:47,120][52833] Updated weights for policy 0, policy_version 13570 (0.0008) -[2023-10-15 15:20:47,488][52833] Updated weights for policy 0, policy_version 13580 (0.0008) -[2023-10-15 15:20:47,857][52833] Updated weights for policy 0, policy_version 13590 (0.0009) -[2023-10-15 15:20:48,229][52833] Updated weights for policy 0, policy_version 13600 (0.0008) -[2023-10-15 15:20:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 27885568. Throughput: 0: 1790.6, 1: 1800.0. Samples: 6972466. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-15 15:20:48,441][51532] Avg episode reward: [(0, '17.770'), (1, '23.420')] -[2023-10-15 15:20:48,442][52518] Saving new best policy, reward=23.420! -[2023-10-15 15:20:49,415][52866] Updated weights for policy 1, policy_version 13640 (0.0008) -[2023-10-15 15:20:49,793][52866] Updated weights for policy 1, policy_version 13650 (0.0010) -[2023-10-15 15:20:50,152][52866] Updated weights for policy 1, policy_version 13660 (0.0008) -[2023-10-15 15:20:51,958][52833] Updated weights for policy 0, policy_version 13610 (0.0007) -[2023-10-15 15:20:52,323][52833] Updated weights for policy 0, policy_version 13620 (0.0007) -[2023-10-15 15:20:52,685][52833] Updated weights for policy 0, policy_version 13630 (0.0007) -[2023-10-15 15:20:53,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 27951104. Throughput: 0: 1806.2, 1: 1800.4. Samples: 6994546. Policy #0 lag: (min: 13.0, avg: 13.4, max: 27.0) -[2023-10-15 15:20:53,442][51532] Avg episode reward: [(0, '17.120'), (1, '21.830')] -[2023-10-15 15:20:53,972][52866] Updated weights for policy 1, policy_version 13670 (0.0008) -[2023-10-15 15:20:54,342][52866] Updated weights for policy 1, policy_version 13680 (0.0008) -[2023-10-15 15:20:54,716][52866] Updated weights for policy 1, policy_version 13690 (0.0008) -[2023-10-15 15:20:56,347][52833] Updated weights for policy 0, policy_version 13640 (0.0008) -[2023-10-15 15:20:56,712][52833] Updated weights for policy 0, policy_version 13650 (0.0008) -[2023-10-15 15:20:57,076][52833] Updated weights for policy 0, policy_version 13660 (0.0008) -[2023-10-15 15:20:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 28016640. Throughput: 0: 1793.5, 1: 1803.0. Samples: 7015898. Policy #0 lag: (min: 13.0, avg: 13.4, max: 27.0) -[2023-10-15 15:20:58,442][51532] Avg episode reward: [(0, '15.850'), (1, '21.510')] -[2023-10-15 15:20:58,514][52866] Updated weights for policy 1, policy_version 13700 (0.0008) -[2023-10-15 15:20:58,881][52866] Updated weights for policy 1, policy_version 13710 (0.0009) -[2023-10-15 15:20:59,240][52866] Updated weights for policy 1, policy_version 13720 (0.0008) -[2023-10-15 15:21:00,847][52833] Updated weights for policy 0, policy_version 13670 (0.0007) -[2023-10-15 15:21:01,208][52833] Updated weights for policy 0, policy_version 13680 (0.0009) -[2023-10-15 15:21:01,584][52833] Updated weights for policy 0, policy_version 13690 (0.0008) -[2023-10-15 15:21:02,932][52866] Updated weights for policy 1, policy_version 13730 (0.0010) -[2023-10-15 15:21:03,295][52866] Updated weights for policy 1, policy_version 13740 (0.0010) -[2023-10-15 15:21:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 28082176. Throughput: 0: 1805.7, 1: 1795.0. Samples: 7026870. Policy #0 lag: (min: 13.0, avg: 13.4, max: 27.0) -[2023-10-15 15:21:03,442][51532] Avg episode reward: [(0, '16.300'), (1, '20.540')] -[2023-10-15 15:21:03,674][52866] Updated weights for policy 1, policy_version 13750 (0.0010) -[2023-10-15 15:21:04,038][52866] Updated weights for policy 1, policy_version 13760 (0.0008) -[2023-10-15 15:21:05,212][52833] Updated weights for policy 0, policy_version 13700 (0.0008) -[2023-10-15 15:21:05,574][52833] Updated weights for policy 0, policy_version 13710 (0.0009) -[2023-10-15 15:21:05,950][52833] Updated weights for policy 0, policy_version 13720 (0.0009) -[2023-10-15 15:21:07,861][52866] Updated weights for policy 1, policy_version 13770 (0.0008) -[2023-10-15 15:21:08,230][52866] Updated weights for policy 1, policy_version 13780 (0.0007) -[2023-10-15 15:21:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 28147712. Throughput: 0: 1801.6, 1: 1800.4. Samples: 7048298. Policy #0 lag: (min: 2.0, avg: 2.0, max: 6.0) -[2023-10-15 15:21:08,441][51532] Avg episode reward: [(0, '17.330'), (1, '21.450')] -[2023-10-15 15:21:08,596][52866] Updated weights for policy 1, policy_version 13790 (0.0007) -[2023-10-15 15:21:09,727][52833] Updated weights for policy 0, policy_version 13730 (0.0009) -[2023-10-15 15:21:10,096][52833] Updated weights for policy 0, policy_version 13740 (0.0010) -[2023-10-15 15:21:10,472][52833] Updated weights for policy 0, policy_version 13750 (0.0010) -[2023-10-15 15:21:10,847][52833] Updated weights for policy 0, policy_version 13760 (0.0011) -[2023-10-15 15:21:12,203][52866] Updated weights for policy 1, policy_version 13800 (0.0007) -[2023-10-15 15:21:12,574][52866] Updated weights for policy 1, policy_version 13810 (0.0007) -[2023-10-15 15:21:12,944][52866] Updated weights for policy 1, policy_version 13820 (0.0007) -[2023-10-15 15:21:13,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 28246016. Throughput: 0: 1802.4, 1: 1798.7. Samples: 7069674. Policy #0 lag: (min: 2.0, avg: 2.0, max: 6.0) -[2023-10-15 15:21:13,442][51532] Avg episode reward: [(0, '18.680'), (1, '19.590')] -[2023-10-15 15:21:14,723][52833] Updated weights for policy 0, policy_version 13770 (0.0009) -[2023-10-15 15:21:15,089][52833] Updated weights for policy 0, policy_version 13780 (0.0007) -[2023-10-15 15:21:15,470][52833] Updated weights for policy 0, policy_version 13790 (0.0007) -[2023-10-15 15:21:16,633][52866] Updated weights for policy 1, policy_version 13830 (0.0009) -[2023-10-15 15:21:17,008][52866] Updated weights for policy 1, policy_version 13840 (0.0009) -[2023-10-15 15:21:17,374][52866] Updated weights for policy 1, policy_version 13850 (0.0011) -[2023-10-15 15:21:18,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 28311552. Throughput: 0: 1802.9, 1: 1795.9. Samples: 7080644. Policy #0 lag: (min: 2.0, avg: 2.0, max: 6.0) -[2023-10-15 15:21:18,442][51532] Avg episode reward: [(0, '20.040'), (1, '19.840')] -[2023-10-15 15:21:19,258][52833] Updated weights for policy 0, policy_version 13800 (0.0007) -[2023-10-15 15:21:19,627][52833] Updated weights for policy 0, policy_version 13810 (0.0008) -[2023-10-15 15:21:20,004][52833] Updated weights for policy 0, policy_version 13820 (0.0008) -[2023-10-15 15:21:20,966][52866] Updated weights for policy 1, policy_version 13860 (0.0009) -[2023-10-15 15:21:21,330][52866] Updated weights for policy 1, policy_version 13870 (0.0008) -[2023-10-15 15:21:21,702][52866] Updated weights for policy 1, policy_version 13880 (0.0007) -[2023-10-15 15:21:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 28377088. Throughput: 0: 1794.7, 1: 1796.9. Samples: 7101718. Policy #0 lag: (min: 11.0, avg: 18.1, max: 43.0) -[2023-10-15 15:21:23,442][51532] Avg episode reward: [(0, '19.960'), (1, '20.380')] -[2023-10-15 15:21:23,917][52833] Updated weights for policy 0, policy_version 13830 (0.0007) -[2023-10-15 15:21:24,295][52833] Updated weights for policy 0, policy_version 13840 (0.0009) -[2023-10-15 15:21:24,670][52833] Updated weights for policy 0, policy_version 13850 (0.0011) -[2023-10-15 15:21:25,323][52866] Updated weights for policy 1, policy_version 13890 (0.0008) -[2023-10-15 15:21:25,689][52866] Updated weights for policy 1, policy_version 13900 (0.0011) -[2023-10-15 15:21:26,064][52866] Updated weights for policy 1, policy_version 13910 (0.0009) -[2023-10-15 15:21:26,428][52866] Updated weights for policy 1, policy_version 13920 (0.0008) -[2023-10-15 15:21:28,386][52833] Updated weights for policy 0, policy_version 13860 (0.0009) -[2023-10-15 15:21:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 28442624. Throughput: 0: 1803.2, 1: 1795.3. Samples: 7124398. Policy #0 lag: (min: 11.0, avg: 18.1, max: 43.0) -[2023-10-15 15:21:28,441][51532] Avg episode reward: [(0, '18.320'), (1, '20.430')] -[2023-10-15 15:21:28,757][52833] Updated weights for policy 0, policy_version 13870 (0.0007) -[2023-10-15 15:21:29,131][52833] Updated weights for policy 0, policy_version 13880 (0.0010) -[2023-10-15 15:21:30,111][52866] Updated weights for policy 1, policy_version 13930 (0.0008) -[2023-10-15 15:21:30,485][52866] Updated weights for policy 1, policy_version 13940 (0.0007) -[2023-10-15 15:21:30,846][52866] Updated weights for policy 1, policy_version 13950 (0.0007) -[2023-10-15 15:21:32,877][52833] Updated weights for policy 0, policy_version 13890 (0.0009) -[2023-10-15 15:21:33,249][52833] Updated weights for policy 0, policy_version 13900 (0.0007) -[2023-10-15 15:21:33,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 28508160. Throughput: 0: 1798.0, 1: 1799.3. Samples: 7134348. Policy #0 lag: (min: 11.0, avg: 18.1, max: 43.0) -[2023-10-15 15:21:33,441][51532] Avg episode reward: [(0, '18.540'), (1, '20.350')] -[2023-10-15 15:21:33,623][52833] Updated weights for policy 0, policy_version 13910 (0.0007) -[2023-10-15 15:21:33,997][52833] Updated weights for policy 0, policy_version 13920 (0.0009) -[2023-10-15 15:21:34,665][52866] Updated weights for policy 1, policy_version 13960 (0.0009) -[2023-10-15 15:21:35,041][52866] Updated weights for policy 1, policy_version 13970 (0.0010) -[2023-10-15 15:21:35,405][52866] Updated weights for policy 1, policy_version 13980 (0.0010) -[2023-10-15 15:21:37,726][52833] Updated weights for policy 0, policy_version 13930 (0.0010) -[2023-10-15 15:21:38,091][52833] Updated weights for policy 0, policy_version 13940 (0.0009) -[2023-10-15 15:21:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 28573696. Throughput: 0: 1801.6, 1: 1802.4. Samples: 7156724. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 15:21:38,442][51532] Avg episode reward: [(0, '18.120'), (1, '19.680')] -[2023-10-15 15:21:38,475][52833] Updated weights for policy 0, policy_version 13950 (0.0008) -[2023-10-15 15:21:39,278][52866] Updated weights for policy 1, policy_version 13990 (0.0008) -[2023-10-15 15:21:39,641][52866] Updated weights for policy 1, policy_version 14000 (0.0008) -[2023-10-15 15:21:40,022][52866] Updated weights for policy 1, policy_version 14010 (0.0009) -[2023-10-15 15:21:42,150][52833] Updated weights for policy 0, policy_version 13960 (0.0008) -[2023-10-15 15:21:42,528][52833] Updated weights for policy 0, policy_version 13970 (0.0008) -[2023-10-15 15:21:42,896][52833] Updated weights for policy 0, policy_version 13980 (0.0007) -[2023-10-15 15:21:43,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 28672000. Throughput: 0: 1796.8, 1: 1803.4. Samples: 7177910. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 15:21:43,442][51532] Avg episode reward: [(0, '18.370'), (1, '19.690')] -[2023-10-15 15:21:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000014016_14352384.pth... -[2023-10-15 15:21:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000013984_14319616.pth... -[2023-10-15 15:21:43,480][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000012352_12648448.pth -[2023-10-15 15:21:43,492][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000012288_12582912.pth -[2023-10-15 15:21:43,784][52866] Updated weights for policy 1, policy_version 14020 (0.0007) -[2023-10-15 15:21:44,166][52866] Updated weights for policy 1, policy_version 14030 (0.0008) -[2023-10-15 15:21:44,524][52866] Updated weights for policy 1, policy_version 14040 (0.0007) -[2023-10-15 15:21:46,700][52833] Updated weights for policy 0, policy_version 13990 (0.0009) -[2023-10-15 15:21:47,073][52833] Updated weights for policy 0, policy_version 14000 (0.0008) -[2023-10-15 15:21:47,441][52833] Updated weights for policy 0, policy_version 14010 (0.0007) -[2023-10-15 15:21:48,416][52866] Updated weights for policy 1, policy_version 14050 (0.0009) -[2023-10-15 15:21:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 28737536. Throughput: 0: 1796.7, 1: 1804.5. Samples: 7188924. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:21:48,441][51532] Avg episode reward: [(0, '18.580'), (1, '20.470')] -[2023-10-15 15:21:48,789][52866] Updated weights for policy 1, policy_version 14060 (0.0011) -[2023-10-15 15:21:49,152][52866] Updated weights for policy 1, policy_version 14070 (0.0009) -[2023-10-15 15:21:49,514][52866] Updated weights for policy 1, policy_version 14080 (0.0009) -[2023-10-15 15:21:51,080][52833] Updated weights for policy 0, policy_version 14020 (0.0009) -[2023-10-15 15:21:51,450][52833] Updated weights for policy 0, policy_version 14030 (0.0010) -[2023-10-15 15:21:51,821][52833] Updated weights for policy 0, policy_version 14040 (0.0007) -[2023-10-15 15:21:53,353][52866] Updated weights for policy 1, policy_version 14090 (0.0007) -[2023-10-15 15:21:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 28803072. Throughput: 0: 1793.1, 1: 1803.4. Samples: 7210144. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:21:53,442][51532] Avg episode reward: [(0, '19.440'), (1, '20.790')] -[2023-10-15 15:21:53,721][52866] Updated weights for policy 1, policy_version 14100 (0.0008) -[2023-10-15 15:21:54,092][52866] Updated weights for policy 1, policy_version 14110 (0.0007) -[2023-10-15 15:21:55,500][52833] Updated weights for policy 0, policy_version 14050 (0.0007) -[2023-10-15 15:21:55,868][52833] Updated weights for policy 0, policy_version 14060 (0.0010) -[2023-10-15 15:21:56,229][52833] Updated weights for policy 0, policy_version 14070 (0.0008) -[2023-10-15 15:21:56,602][52833] Updated weights for policy 0, policy_version 14080 (0.0010) -[2023-10-15 15:21:57,726][52866] Updated weights for policy 1, policy_version 14120 (0.0010) -[2023-10-15 15:21:58,096][52866] Updated weights for policy 1, policy_version 14130 (0.0009) -[2023-10-15 15:21:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 28868608. Throughput: 0: 1782.8, 1: 1816.7. Samples: 7231652. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:21:58,441][51532] Avg episode reward: [(0, '18.870'), (1, '20.810')] -[2023-10-15 15:21:58,457][52866] Updated weights for policy 1, policy_version 14140 (0.0010) -[2023-10-15 15:22:00,374][52833] Updated weights for policy 0, policy_version 14090 (0.0010) -[2023-10-15 15:22:00,738][52833] Updated weights for policy 0, policy_version 14100 (0.0009) -[2023-10-15 15:22:01,106][52833] Updated weights for policy 0, policy_version 14110 (0.0008) -[2023-10-15 15:22:02,276][52866] Updated weights for policy 1, policy_version 14150 (0.0009) -[2023-10-15 15:22:02,662][52866] Updated weights for policy 1, policy_version 14160 (0.0007) -[2023-10-15 15:22:03,016][52866] Updated weights for policy 1, policy_version 14170 (0.0007) -[2023-10-15 15:22:03,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 28966912. Throughput: 0: 1794.3, 1: 1799.0. Samples: 7242340. Policy #0 lag: (min: 17.0, avg: 18.9, max: 46.0) -[2023-10-15 15:22:03,442][51532] Avg episode reward: [(0, '19.900'), (1, '22.000')] -[2023-10-15 15:22:04,901][52833] Updated weights for policy 0, policy_version 14120 (0.0009) -[2023-10-15 15:22:05,277][52833] Updated weights for policy 0, policy_version 14130 (0.0007) -[2023-10-15 15:22:05,651][52833] Updated weights for policy 0, policy_version 14140 (0.0010) -[2023-10-15 15:22:06,667][52866] Updated weights for policy 1, policy_version 14180 (0.0008) -[2023-10-15 15:22:07,041][52866] Updated weights for policy 1, policy_version 14190 (0.0010) -[2023-10-15 15:22:07,404][52866] Updated weights for policy 1, policy_version 14200 (0.0010) -[2023-10-15 15:22:08,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 29032448. Throughput: 0: 1787.6, 1: 1816.4. Samples: 7263898. Policy #0 lag: (min: 17.0, avg: 18.9, max: 46.0) -[2023-10-15 15:22:08,442][51532] Avg episode reward: [(0, '19.300'), (1, '22.100')] -[2023-10-15 15:22:09,482][52833] Updated weights for policy 0, policy_version 14150 (0.0007) -[2023-10-15 15:22:09,855][52833] Updated weights for policy 0, policy_version 14160 (0.0009) -[2023-10-15 15:22:10,229][52833] Updated weights for policy 0, policy_version 14170 (0.0007) -[2023-10-15 15:22:11,056][52866] Updated weights for policy 1, policy_version 14210 (0.0010) -[2023-10-15 15:22:11,420][52866] Updated weights for policy 1, policy_version 14220 (0.0009) -[2023-10-15 15:22:11,785][52866] Updated weights for policy 1, policy_version 14230 (0.0007) -[2023-10-15 15:22:12,146][52866] Updated weights for policy 1, policy_version 14240 (0.0007) -[2023-10-15 15:22:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 29097984. Throughput: 0: 1789.1, 1: 1792.4. Samples: 7285566. Policy #0 lag: (min: 17.0, avg: 18.9, max: 46.0) -[2023-10-15 15:22:13,441][51532] Avg episode reward: [(0, '20.100'), (1, '23.830')] -[2023-10-15 15:22:13,454][52518] Saving new best policy, reward=23.830! -[2023-10-15 15:22:13,923][52833] Updated weights for policy 0, policy_version 14180 (0.0010) -[2023-10-15 15:22:14,290][52833] Updated weights for policy 0, policy_version 14190 (0.0011) -[2023-10-15 15:22:14,649][52833] Updated weights for policy 0, policy_version 14200 (0.0011) -[2023-10-15 15:22:15,952][52866] Updated weights for policy 1, policy_version 14250 (0.0007) -[2023-10-15 15:22:16,324][52866] Updated weights for policy 1, policy_version 14260 (0.0007) -[2023-10-15 15:22:16,692][52866] Updated weights for policy 1, policy_version 14270 (0.0007) -[2023-10-15 15:22:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 29163520. Throughput: 0: 1786.5, 1: 1809.2. Samples: 7296158. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 15:22:18,442][51532] Avg episode reward: [(0, '20.930'), (1, '22.080')] -[2023-10-15 15:22:18,463][52833] Updated weights for policy 0, policy_version 14210 (0.0009) -[2023-10-15 15:22:18,836][52833] Updated weights for policy 0, policy_version 14220 (0.0009) -[2023-10-15 15:22:19,204][52833] Updated weights for policy 0, policy_version 14230 (0.0008) -[2023-10-15 15:22:19,569][52833] Updated weights for policy 0, policy_version 14240 (0.0007) -[2023-10-15 15:22:20,300][52866] Updated weights for policy 1, policy_version 14280 (0.0008) -[2023-10-15 15:22:20,669][52866] Updated weights for policy 1, policy_version 14290 (0.0008) -[2023-10-15 15:22:21,040][52866] Updated weights for policy 1, policy_version 14300 (0.0010) -[2023-10-15 15:22:23,271][52833] Updated weights for policy 0, policy_version 14250 (0.0010) -[2023-10-15 15:22:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 29229056. Throughput: 0: 1793.9, 1: 1788.5. Samples: 7317932. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 15:22:23,441][51532] Avg episode reward: [(0, '20.290'), (1, '22.720')] -[2023-10-15 15:22:23,637][52833] Updated weights for policy 0, policy_version 14260 (0.0009) -[2023-10-15 15:22:24,003][52833] Updated weights for policy 0, policy_version 14270 (0.0011) -[2023-10-15 15:22:24,862][52866] Updated weights for policy 1, policy_version 14310 (0.0011) -[2023-10-15 15:22:25,223][52866] Updated weights for policy 1, policy_version 14320 (0.0010) -[2023-10-15 15:22:25,593][52866] Updated weights for policy 1, policy_version 14330 (0.0010) -[2023-10-15 15:22:27,854][52833] Updated weights for policy 0, policy_version 14280 (0.0008) -[2023-10-15 15:22:28,233][52833] Updated weights for policy 0, policy_version 14290 (0.0008) -[2023-10-15 15:22:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 29294592. Throughput: 0: 1812.0, 1: 1787.4. Samples: 7339882. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-15 15:22:28,441][51532] Avg episode reward: [(0, '18.920'), (1, '23.690')] -[2023-10-15 15:22:28,600][52833] Updated weights for policy 0, policy_version 14300 (0.0007) -[2023-10-15 15:22:29,539][52866] Updated weights for policy 1, policy_version 14340 (0.0010) -[2023-10-15 15:22:29,899][52866] Updated weights for policy 1, policy_version 14350 (0.0007) -[2023-10-15 15:22:30,268][52866] Updated weights for policy 1, policy_version 14360 (0.0007) -[2023-10-15 15:22:32,400][52833] Updated weights for policy 0, policy_version 14310 (0.0010) -[2023-10-15 15:22:32,771][52833] Updated weights for policy 0, policy_version 14320 (0.0009) -[2023-10-15 15:22:33,133][52833] Updated weights for policy 0, policy_version 14330 (0.0010) -[2023-10-15 15:22:33,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 29392896. Throughput: 0: 1795.2, 1: 1787.9. Samples: 7350164. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) -[2023-10-15 15:22:33,442][51532] Avg episode reward: [(0, '19.630'), (1, '23.140')] -[2023-10-15 15:22:33,915][52866] Updated weights for policy 1, policy_version 14370 (0.0009) -[2023-10-15 15:22:34,289][52866] Updated weights for policy 1, policy_version 14380 (0.0009) -[2023-10-15 15:22:34,651][52866] Updated weights for policy 1, policy_version 14390 (0.0008) -[2023-10-15 15:22:35,019][52866] Updated weights for policy 1, policy_version 14400 (0.0008) -[2023-10-15 15:22:36,756][52833] Updated weights for policy 0, policy_version 14340 (0.0011) -[2023-10-15 15:22:37,125][52833] Updated weights for policy 0, policy_version 14350 (0.0008) -[2023-10-15 15:22:37,503][52833] Updated weights for policy 0, policy_version 14360 (0.0008) -[2023-10-15 15:22:38,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 29458432. Throughput: 0: 1809.6, 1: 1798.6. Samples: 7372510. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) -[2023-10-15 15:22:38,442][51532] Avg episode reward: [(0, '18.440'), (1, '23.790')] -[2023-10-15 15:22:38,581][52866] Updated weights for policy 1, policy_version 14410 (0.0009) -[2023-10-15 15:22:38,947][52866] Updated weights for policy 1, policy_version 14420 (0.0008) -[2023-10-15 15:22:39,315][52866] Updated weights for policy 1, policy_version 14430 (0.0007) -[2023-10-15 15:22:41,213][52833] Updated weights for policy 0, policy_version 14370 (0.0008) -[2023-10-15 15:22:41,595][52833] Updated weights for policy 0, policy_version 14380 (0.0009) -[2023-10-15 15:22:41,969][52833] Updated weights for policy 0, policy_version 14390 (0.0008) -[2023-10-15 15:22:42,339][52833] Updated weights for policy 0, policy_version 14400 (0.0008) -[2023-10-15 15:22:43,003][52866] Updated weights for policy 1, policy_version 14440 (0.0009) -[2023-10-15 15:22:43,378][52866] Updated weights for policy 1, policy_version 14450 (0.0008) -[2023-10-15 15:22:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 29523968. Throughput: 0: 1791.7, 1: 1812.8. Samples: 7393856. Policy #0 lag: (min: 18.0, avg: 38.1, max: 40.0) -[2023-10-15 15:22:43,442][51532] Avg episode reward: [(0, '18.130'), (1, '23.430')] -[2023-10-15 15:22:43,742][52866] Updated weights for policy 1, policy_version 14460 (0.0008) -[2023-10-15 15:22:45,968][52833] Updated weights for policy 0, policy_version 14410 (0.0007) -[2023-10-15 15:22:46,337][52833] Updated weights for policy 0, policy_version 14420 (0.0007) -[2023-10-15 15:22:46,717][52833] Updated weights for policy 0, policy_version 14430 (0.0010) -[2023-10-15 15:22:47,374][52866] Updated weights for policy 1, policy_version 14470 (0.0009) -[2023-10-15 15:22:47,751][52866] Updated weights for policy 1, policy_version 14480 (0.0009) -[2023-10-15 15:22:48,119][52866] Updated weights for policy 1, policy_version 14490 (0.0008) -[2023-10-15 15:22:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 29622272. Throughput: 0: 1812.5, 1: 1811.5. Samples: 7405418. Policy #0 lag: (min: 18.0, avg: 38.1, max: 40.0) -[2023-10-15 15:22:48,441][51532] Avg episode reward: [(0, '19.880'), (1, '23.320')] -[2023-10-15 15:22:50,581][52833] Updated weights for policy 0, policy_version 14440 (0.0008) -[2023-10-15 15:22:50,948][52833] Updated weights for policy 0, policy_version 14450 (0.0009) -[2023-10-15 15:22:51,326][52833] Updated weights for policy 0, policy_version 14460 (0.0009) -[2023-10-15 15:22:51,856][52866] Updated weights for policy 1, policy_version 14500 (0.0011) -[2023-10-15 15:22:52,217][52866] Updated weights for policy 1, policy_version 14510 (0.0010) -[2023-10-15 15:22:52,582][52866] Updated weights for policy 1, policy_version 14520 (0.0008) -[2023-10-15 15:22:53,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 29687808. Throughput: 0: 1796.0, 1: 1811.4. Samples: 7426232. Policy #0 lag: (min: 18.0, avg: 38.1, max: 40.0) -[2023-10-15 15:22:53,441][51532] Avg episode reward: [(0, '20.960'), (1, '23.990')] -[2023-10-15 15:22:53,442][52518] Saving new best policy, reward=23.990! -[2023-10-15 15:22:54,981][52833] Updated weights for policy 0, policy_version 14470 (0.0008) -[2023-10-15 15:22:55,353][52833] Updated weights for policy 0, policy_version 14480 (0.0008) -[2023-10-15 15:22:55,727][52833] Updated weights for policy 0, policy_version 14490 (0.0007) -[2023-10-15 15:22:56,147][52866] Updated weights for policy 1, policy_version 14530 (0.0010) -[2023-10-15 15:22:56,511][52866] Updated weights for policy 1, policy_version 14540 (0.0008) -[2023-10-15 15:22:56,883][52866] Updated weights for policy 1, policy_version 14550 (0.0008) -[2023-10-15 15:22:57,242][52866] Updated weights for policy 1, policy_version 14560 (0.0007) -[2023-10-15 15:22:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 29753344. Throughput: 0: 1796.1, 1: 1808.3. Samples: 7447764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:22:58,442][51532] Avg episode reward: [(0, '21.060'), (1, '23.210')] -[2023-10-15 15:22:59,384][52833] Updated weights for policy 0, policy_version 14500 (0.0007) -[2023-10-15 15:22:59,750][52833] Updated weights for policy 0, policy_version 14510 (0.0009) -[2023-10-15 15:23:00,117][52833] Updated weights for policy 0, policy_version 14520 (0.0010) -[2023-10-15 15:23:01,022][52866] Updated weights for policy 1, policy_version 14570 (0.0009) -[2023-10-15 15:23:01,382][52866] Updated weights for policy 1, policy_version 14580 (0.0009) -[2023-10-15 15:23:01,756][52866] Updated weights for policy 1, policy_version 14590 (0.0011) -[2023-10-15 15:23:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29818880. Throughput: 0: 1798.6, 1: 1813.6. Samples: 7458706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:23:03,441][51532] Avg episode reward: [(0, '20.870'), (1, '22.040')] -[2023-10-15 15:23:03,935][52833] Updated weights for policy 0, policy_version 14530 (0.0010) -[2023-10-15 15:23:04,306][52833] Updated weights for policy 0, policy_version 14540 (0.0009) -[2023-10-15 15:23:04,671][52833] Updated weights for policy 0, policy_version 14550 (0.0008) -[2023-10-15 15:23:05,049][52833] Updated weights for policy 0, policy_version 14560 (0.0007) -[2023-10-15 15:23:05,457][52866] Updated weights for policy 1, policy_version 14600 (0.0008) -[2023-10-15 15:23:05,822][52866] Updated weights for policy 1, policy_version 14610 (0.0008) -[2023-10-15 15:23:06,187][52866] Updated weights for policy 1, policy_version 14620 (0.0009) -[2023-10-15 15:23:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 29884416. Throughput: 0: 1794.3, 1: 1812.2. Samples: 7480226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:23:08,442][51532] Avg episode reward: [(0, '19.310'), (1, '23.830')] -[2023-10-15 15:23:08,862][52833] Updated weights for policy 0, policy_version 14570 (0.0009) -[2023-10-15 15:23:09,228][52833] Updated weights for policy 0, policy_version 14580 (0.0009) -[2023-10-15 15:23:09,601][52833] Updated weights for policy 0, policy_version 14590 (0.0009) -[2023-10-15 15:23:10,012][52866] Updated weights for policy 1, policy_version 14630 (0.0008) -[2023-10-15 15:23:10,370][52866] Updated weights for policy 1, policy_version 14640 (0.0008) -[2023-10-15 15:23:10,731][52866] Updated weights for policy 1, policy_version 14650 (0.0007) -[2023-10-15 15:23:13,274][52833] Updated weights for policy 0, policy_version 14600 (0.0010) -[2023-10-15 15:23:13,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 29949952. Throughput: 0: 1800.6, 1: 1815.6. Samples: 7502610. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:23:13,442][51532] Avg episode reward: [(0, '17.560'), (1, '23.790')] -[2023-10-15 15:23:13,643][52833] Updated weights for policy 0, policy_version 14610 (0.0010) -[2023-10-15 15:23:14,011][52833] Updated weights for policy 0, policy_version 14620 (0.0008) -[2023-10-15 15:23:14,413][52866] Updated weights for policy 1, policy_version 14660 (0.0007) -[2023-10-15 15:23:14,780][52866] Updated weights for policy 1, policy_version 14670 (0.0007) -[2023-10-15 15:23:15,149][52866] Updated weights for policy 1, policy_version 14680 (0.0009) -[2023-10-15 15:23:17,848][52833] Updated weights for policy 0, policy_version 14630 (0.0008) -[2023-10-15 15:23:18,221][52833] Updated weights for policy 0, policy_version 14640 (0.0010) -[2023-10-15 15:23:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 30015488. Throughput: 0: 1791.1, 1: 1813.9. Samples: 7512386. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:23:18,442][51532] Avg episode reward: [(0, '18.700'), (1, '24.090')] -[2023-10-15 15:23:18,443][52518] Saving new best policy, reward=24.090! -[2023-10-15 15:23:18,594][52833] Updated weights for policy 0, policy_version 14650 (0.0008) -[2023-10-15 15:23:18,886][52866] Updated weights for policy 1, policy_version 14690 (0.0010) -[2023-10-15 15:23:19,249][52866] Updated weights for policy 1, policy_version 14700 (0.0011) -[2023-10-15 15:23:19,619][52866] Updated weights for policy 1, policy_version 14710 (0.0010) -[2023-10-15 15:23:19,988][52866] Updated weights for policy 1, policy_version 14720 (0.0010) -[2023-10-15 15:23:22,292][52833] Updated weights for policy 0, policy_version 14660 (0.0008) -[2023-10-15 15:23:22,669][52833] Updated weights for policy 0, policy_version 14670 (0.0011) -[2023-10-15 15:23:23,029][52833] Updated weights for policy 0, policy_version 14680 (0.0009) -[2023-10-15 15:23:23,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 30113792. Throughput: 0: 1799.7, 1: 1805.9. Samples: 7534760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:23:23,441][51532] Avg episode reward: [(0, '18.040'), (1, '23.170')] -[2023-10-15 15:23:23,821][52866] Updated weights for policy 1, policy_version 14730 (0.0009) -[2023-10-15 15:23:24,199][52866] Updated weights for policy 1, policy_version 14740 (0.0007) -[2023-10-15 15:23:24,563][52866] Updated weights for policy 1, policy_version 14750 (0.0007) -[2023-10-15 15:23:26,641][52833] Updated weights for policy 0, policy_version 14690 (0.0010) -[2023-10-15 15:23:27,015][52833] Updated weights for policy 0, policy_version 14700 (0.0010) -[2023-10-15 15:23:27,384][52833] Updated weights for policy 0, policy_version 14710 (0.0008) -[2023-10-15 15:23:27,761][52833] Updated weights for policy 0, policy_version 14720 (0.0009) -[2023-10-15 15:23:28,314][52866] Updated weights for policy 1, policy_version 14760 (0.0009) -[2023-10-15 15:23:28,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 30179328. Throughput: 0: 1795.8, 1: 1802.8. Samples: 7555792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:23:28,441][51532] Avg episode reward: [(0, '19.520'), (1, '25.040')] -[2023-10-15 15:23:28,686][52866] Updated weights for policy 1, policy_version 14770 (0.0009) -[2023-10-15 15:23:29,059][52866] Updated weights for policy 1, policy_version 14780 (0.0009) -[2023-10-15 15:23:29,209][52518] Saving new best policy, reward=25.040! -[2023-10-15 15:23:31,578][52833] Updated weights for policy 0, policy_version 14730 (0.0011) -[2023-10-15 15:23:31,959][52833] Updated weights for policy 0, policy_version 14740 (0.0008) -[2023-10-15 15:23:32,323][52833] Updated weights for policy 0, policy_version 14750 (0.0009) -[2023-10-15 15:23:33,023][52866] Updated weights for policy 1, policy_version 14790 (0.0009) -[2023-10-15 15:23:33,413][52866] Updated weights for policy 1, policy_version 14800 (0.0007) -[2023-10-15 15:23:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 30244864. Throughput: 0: 1798.3, 1: 1789.3. Samples: 7566858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:23:33,442][51532] Avg episode reward: [(0, '19.250'), (1, '24.720')] -[2023-10-15 15:23:33,771][52866] Updated weights for policy 1, policy_version 14810 (0.0008) -[2023-10-15 15:23:36,238][52833] Updated weights for policy 0, policy_version 14760 (0.0007) -[2023-10-15 15:23:36,603][52833] Updated weights for policy 0, policy_version 14770 (0.0011) -[2023-10-15 15:23:36,977][52833] Updated weights for policy 0, policy_version 14780 (0.0009) -[2023-10-15 15:23:37,615][52866] Updated weights for policy 1, policy_version 14820 (0.0010) -[2023-10-15 15:23:37,986][52866] Updated weights for policy 1, policy_version 14830 (0.0007) -[2023-10-15 15:23:38,349][52866] Updated weights for policy 1, policy_version 14840 (0.0009) -[2023-10-15 15:23:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 30310400. Throughput: 0: 1794.1, 1: 1797.8. Samples: 7587868. Policy #0 lag: (min: 14.0, avg: 17.1, max: 46.0) -[2023-10-15 15:23:38,441][51532] Avg episode reward: [(0, '20.020'), (1, '22.630')] -[2023-10-15 15:23:40,762][52833] Updated weights for policy 0, policy_version 14790 (0.0007) -[2023-10-15 15:23:41,149][52833] Updated weights for policy 0, policy_version 14800 (0.0008) -[2023-10-15 15:23:41,525][52833] Updated weights for policy 0, policy_version 14810 (0.0009) -[2023-10-15 15:23:41,987][52866] Updated weights for policy 1, policy_version 14850 (0.0010) -[2023-10-15 15:23:42,352][52866] Updated weights for policy 1, policy_version 14860 (0.0007) -[2023-10-15 15:23:42,707][52866] Updated weights for policy 1, policy_version 14870 (0.0007) -[2023-10-15 15:23:43,075][52866] Updated weights for policy 1, policy_version 14880 (0.0010) -[2023-10-15 15:23:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 30408704. Throughput: 0: 1783.8, 1: 1796.9. Samples: 7608896. Policy #0 lag: (min: 14.0, avg: 17.1, max: 46.0) -[2023-10-15 15:23:43,442][51532] Avg episode reward: [(0, '19.900'), (1, '23.170')] -[2023-10-15 15:23:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000014816_15171584.pth... -[2023-10-15 15:23:43,453][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000014880_15237120.pth... -[2023-10-15 15:23:43,490][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000013184_13500416.pth -[2023-10-15 15:23:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000013152_13467648.pth -[2023-10-15 15:23:45,187][52833] Updated weights for policy 0, policy_version 14820 (0.0008) -[2023-10-15 15:23:45,563][52833] Updated weights for policy 0, policy_version 14830 (0.0009) -[2023-10-15 15:23:45,925][52833] Updated weights for policy 0, policy_version 14840 (0.0008) -[2023-10-15 15:23:46,774][52866] Updated weights for policy 1, policy_version 14890 (0.0010) -[2023-10-15 15:23:47,134][52866] Updated weights for policy 1, policy_version 14900 (0.0008) -[2023-10-15 15:23:47,504][52866] Updated weights for policy 1, policy_version 14910 (0.0010) -[2023-10-15 15:23:48,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30474240. Throughput: 0: 1794.5, 1: 1802.2. Samples: 7620558. Policy #0 lag: (min: 14.0, avg: 17.1, max: 46.0) -[2023-10-15 15:23:48,442][51532] Avg episode reward: [(0, '20.220'), (1, '25.080')] -[2023-10-15 15:23:48,443][52518] Saving new best policy, reward=25.080! -[2023-10-15 15:23:49,661][52833] Updated weights for policy 0, policy_version 14850 (0.0008) -[2023-10-15 15:23:50,035][52833] Updated weights for policy 0, policy_version 14860 (0.0007) -[2023-10-15 15:23:50,400][52833] Updated weights for policy 0, policy_version 14870 (0.0009) -[2023-10-15 15:23:50,770][52833] Updated weights for policy 0, policy_version 14880 (0.0010) -[2023-10-15 15:23:51,293][52866] Updated weights for policy 1, policy_version 14920 (0.0010) -[2023-10-15 15:23:51,663][52866] Updated weights for policy 1, policy_version 14930 (0.0008) -[2023-10-15 15:23:52,037][52866] Updated weights for policy 1, policy_version 14940 (0.0007) -[2023-10-15 15:23:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 30539776. Throughput: 0: 1785.2, 1: 1797.6. Samples: 7641454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:23:53,442][51532] Avg episode reward: [(0, '20.100'), (1, '25.430')] -[2023-10-15 15:23:53,443][52518] Saving new best policy, reward=25.430! -[2023-10-15 15:23:54,537][52833] Updated weights for policy 0, policy_version 14890 (0.0008) -[2023-10-15 15:23:54,911][52833] Updated weights for policy 0, policy_version 14900 (0.0008) -[2023-10-15 15:23:55,275][52833] Updated weights for policy 0, policy_version 14910 (0.0007) -[2023-10-15 15:23:55,693][52866] Updated weights for policy 1, policy_version 14950 (0.0008) -[2023-10-15 15:23:56,070][52866] Updated weights for policy 1, policy_version 14960 (0.0008) -[2023-10-15 15:23:56,438][52866] Updated weights for policy 1, policy_version 14970 (0.0009) -[2023-10-15 15:23:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 30605312. Throughput: 0: 1790.0, 1: 1794.2. Samples: 7663898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:23:58,441][51532] Avg episode reward: [(0, '21.790'), (1, '23.900')] -[2023-10-15 15:23:59,066][52833] Updated weights for policy 0, policy_version 14920 (0.0008) -[2023-10-15 15:23:59,435][52833] Updated weights for policy 0, policy_version 14930 (0.0007) -[2023-10-15 15:23:59,802][52833] Updated weights for policy 0, policy_version 14940 (0.0007) -[2023-10-15 15:24:00,168][52866] Updated weights for policy 1, policy_version 14980 (0.0010) -[2023-10-15 15:24:00,549][52866] Updated weights for policy 1, policy_version 14990 (0.0008) -[2023-10-15 15:24:00,907][52866] Updated weights for policy 1, policy_version 15000 (0.0009) -[2023-10-15 15:24:03,401][52833] Updated weights for policy 0, policy_version 14950 (0.0007) -[2023-10-15 15:24:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 30670848. Throughput: 0: 1794.6, 1: 1803.8. Samples: 7674314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:24:03,441][51532] Avg episode reward: [(0, '20.530'), (1, '24.470')] -[2023-10-15 15:24:03,768][52833] Updated weights for policy 0, policy_version 14960 (0.0007) -[2023-10-15 15:24:04,148][52833] Updated weights for policy 0, policy_version 14970 (0.0008) -[2023-10-15 15:24:04,736][52866] Updated weights for policy 1, policy_version 15010 (0.0009) -[2023-10-15 15:24:05,113][52866] Updated weights for policy 1, policy_version 15020 (0.0008) -[2023-10-15 15:24:05,472][52866] Updated weights for policy 1, policy_version 15030 (0.0008) -[2023-10-15 15:24:05,842][52866] Updated weights for policy 1, policy_version 15040 (0.0010) -[2023-10-15 15:24:07,947][52833] Updated weights for policy 0, policy_version 14980 (0.0008) -[2023-10-15 15:24:08,309][52833] Updated weights for policy 0, policy_version 14990 (0.0008) -[2023-10-15 15:24:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 30736384. Throughput: 0: 1797.0, 1: 1791.1. Samples: 7696224. Policy #0 lag: (min: 5.0, avg: 12.9, max: 37.0) -[2023-10-15 15:24:08,442][51532] Avg episode reward: [(0, '21.950'), (1, '24.100')] -[2023-10-15 15:24:08,685][52833] Updated weights for policy 0, policy_version 15000 (0.0007) -[2023-10-15 15:24:08,976][52410] Saving new best policy, reward=21.950! -[2023-10-15 15:24:09,517][52866] Updated weights for policy 1, policy_version 15050 (0.0007) -[2023-10-15 15:24:09,886][52866] Updated weights for policy 1, policy_version 15060 (0.0009) -[2023-10-15 15:24:10,259][52866] Updated weights for policy 1, policy_version 15070 (0.0008) -[2023-10-15 15:24:12,528][52833] Updated weights for policy 0, policy_version 15010 (0.0008) -[2023-10-15 15:24:12,898][52833] Updated weights for policy 0, policy_version 15020 (0.0008) -[2023-10-15 15:24:13,270][52833] Updated weights for policy 0, policy_version 15030 (0.0008) -[2023-10-15 15:24:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 30801920. Throughput: 0: 1814.9, 1: 1792.1. Samples: 7718106. Policy #0 lag: (min: 5.0, avg: 12.9, max: 37.0) -[2023-10-15 15:24:13,442][51532] Avg episode reward: [(0, '21.680'), (1, '23.300')] -[2023-10-15 15:24:13,638][52833] Updated weights for policy 0, policy_version 15040 (0.0010) -[2023-10-15 15:24:14,025][52866] Updated weights for policy 1, policy_version 15080 (0.0008) -[2023-10-15 15:24:14,392][52866] Updated weights for policy 1, policy_version 15090 (0.0007) -[2023-10-15 15:24:14,753][52866] Updated weights for policy 1, policy_version 15100 (0.0007) -[2023-10-15 15:24:17,173][52833] Updated weights for policy 0, policy_version 15050 (0.0007) -[2023-10-15 15:24:17,546][52833] Updated weights for policy 0, policy_version 15060 (0.0010) -[2023-10-15 15:24:17,911][52833] Updated weights for policy 0, policy_version 15070 (0.0010) -[2023-10-15 15:24:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 30900224. Throughput: 0: 1795.9, 1: 1794.4. Samples: 7728420. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:24:18,442][51532] Avg episode reward: [(0, '21.940'), (1, '21.530')] -[2023-10-15 15:24:18,684][52866] Updated weights for policy 1, policy_version 15110 (0.0008) -[2023-10-15 15:24:19,056][52866] Updated weights for policy 1, policy_version 15120 (0.0007) -[2023-10-15 15:24:19,419][52866] Updated weights for policy 1, policy_version 15130 (0.0007) -[2023-10-15 15:24:21,711][52833] Updated weights for policy 0, policy_version 15080 (0.0007) -[2023-10-15 15:24:22,077][52833] Updated weights for policy 0, policy_version 15090 (0.0007) -[2023-10-15 15:24:22,448][52833] Updated weights for policy 0, policy_version 15100 (0.0007) -[2023-10-15 15:24:23,062][52866] Updated weights for policy 1, policy_version 15140 (0.0007) -[2023-10-15 15:24:23,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 30965760. Throughput: 0: 1816.0, 1: 1793.5. Samples: 7750294. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:24:23,442][51532] Avg episode reward: [(0, '21.710'), (1, '20.920')] -[2023-10-15 15:24:23,443][52866] Updated weights for policy 1, policy_version 15150 (0.0010) -[2023-10-15 15:24:23,806][52866] Updated weights for policy 1, policy_version 15160 (0.0010) -[2023-10-15 15:24:26,229][52833] Updated weights for policy 0, policy_version 15110 (0.0008) -[2023-10-15 15:24:26,607][52833] Updated weights for policy 0, policy_version 15120 (0.0008) -[2023-10-15 15:24:26,975][52833] Updated weights for policy 0, policy_version 15130 (0.0008) -[2023-10-15 15:24:27,636][52866] Updated weights for policy 1, policy_version 15170 (0.0009) -[2023-10-15 15:24:27,997][52866] Updated weights for policy 1, policy_version 15180 (0.0009) -[2023-10-15 15:24:28,376][52866] Updated weights for policy 1, policy_version 15190 (0.0008) -[2023-10-15 15:24:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 31031296. Throughput: 0: 1800.5, 1: 1808.0. Samples: 7771278. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:24:28,442][51532] Avg episode reward: [(0, '21.790'), (1, '21.220')] -[2023-10-15 15:24:28,747][52866] Updated weights for policy 1, policy_version 15200 (0.0009) -[2023-10-15 15:24:30,758][52833] Updated weights for policy 0, policy_version 15140 (0.0007) -[2023-10-15 15:24:31,121][52833] Updated weights for policy 0, policy_version 15150 (0.0009) -[2023-10-15 15:24:31,496][52833] Updated weights for policy 0, policy_version 15160 (0.0011) -[2023-10-15 15:24:32,427][52866] Updated weights for policy 1, policy_version 15210 (0.0007) -[2023-10-15 15:24:32,791][52866] Updated weights for policy 1, policy_version 15220 (0.0008) -[2023-10-15 15:24:33,162][52866] Updated weights for policy 1, policy_version 15230 (0.0008) -[2023-10-15 15:24:33,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 31129600. Throughput: 0: 1816.2, 1: 1789.7. Samples: 7782826. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:24:33,442][51532] Avg episode reward: [(0, '20.300'), (1, '20.390')] -[2023-10-15 15:24:35,224][52833] Updated weights for policy 0, policy_version 15170 (0.0010) -[2023-10-15 15:24:35,599][52833] Updated weights for policy 0, policy_version 15180 (0.0011) -[2023-10-15 15:24:35,972][52833] Updated weights for policy 0, policy_version 15190 (0.0009) -[2023-10-15 15:24:36,343][52833] Updated weights for policy 0, policy_version 15200 (0.0008) -[2023-10-15 15:24:36,948][52866] Updated weights for policy 1, policy_version 15240 (0.0008) -[2023-10-15 15:24:37,324][52866] Updated weights for policy 1, policy_version 15250 (0.0009) -[2023-10-15 15:24:37,679][52866] Updated weights for policy 1, policy_version 15260 (0.0007) -[2023-10-15 15:24:38,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 31195136. Throughput: 0: 1796.9, 1: 1808.6. Samples: 7803700. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:24:38,441][51532] Avg episode reward: [(0, '20.850'), (1, '21.680')] -[2023-10-15 15:24:40,015][52833] Updated weights for policy 0, policy_version 15210 (0.0009) -[2023-10-15 15:24:40,388][52833] Updated weights for policy 0, policy_version 15220 (0.0009) -[2023-10-15 15:24:40,759][52833] Updated weights for policy 0, policy_version 15230 (0.0009) -[2023-10-15 15:24:41,401][52866] Updated weights for policy 1, policy_version 15270 (0.0010) -[2023-10-15 15:24:41,768][52866] Updated weights for policy 1, policy_version 15280 (0.0008) -[2023-10-15 15:24:42,144][52866] Updated weights for policy 1, policy_version 15290 (0.0010) -[2023-10-15 15:24:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31260672. Throughput: 0: 1793.2, 1: 1786.8. Samples: 7824998. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:24:43,442][51532] Avg episode reward: [(0, '22.040'), (1, '21.850')] -[2023-10-15 15:24:43,452][52410] Saving new best policy, reward=22.040! -[2023-10-15 15:24:44,575][52833] Updated weights for policy 0, policy_version 15240 (0.0008) -[2023-10-15 15:24:44,938][52833] Updated weights for policy 0, policy_version 15250 (0.0008) -[2023-10-15 15:24:45,306][52833] Updated weights for policy 0, policy_version 15260 (0.0008) -[2023-10-15 15:24:45,923][52866] Updated weights for policy 1, policy_version 15300 (0.0009) -[2023-10-15 15:24:46,287][52866] Updated weights for policy 1, policy_version 15310 (0.0008) -[2023-10-15 15:24:46,669][52866] Updated weights for policy 1, policy_version 15320 (0.0009) -[2023-10-15 15:24:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31326208. Throughput: 0: 1787.1, 1: 1804.6. Samples: 7835938. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:24:48,442][51532] Avg episode reward: [(0, '21.680'), (1, '24.040')] -[2023-10-15 15:24:49,166][52833] Updated weights for policy 0, policy_version 15270 (0.0008) -[2023-10-15 15:24:49,529][52833] Updated weights for policy 0, policy_version 15280 (0.0008) -[2023-10-15 15:24:49,900][52833] Updated weights for policy 0, policy_version 15290 (0.0008) -[2023-10-15 15:24:50,515][52866] Updated weights for policy 1, policy_version 15330 (0.0009) -[2023-10-15 15:24:50,879][52866] Updated weights for policy 1, policy_version 15340 (0.0010) -[2023-10-15 15:24:51,240][52866] Updated weights for policy 1, policy_version 15350 (0.0010) -[2023-10-15 15:24:51,616][52866] Updated weights for policy 1, policy_version 15360 (0.0009) -[2023-10-15 15:24:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 31391744. Throughput: 0: 1786.8, 1: 1783.9. Samples: 7856904. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:24:53,442][51532] Avg episode reward: [(0, '23.050'), (1, '24.380')] -[2023-10-15 15:24:53,582][52833] Updated weights for policy 0, policy_version 15300 (0.0008) -[2023-10-15 15:24:53,949][52833] Updated weights for policy 0, policy_version 15310 (0.0010) -[2023-10-15 15:24:54,324][52833] Updated weights for policy 0, policy_version 15320 (0.0007) -[2023-10-15 15:24:54,612][52410] Saving new best policy, reward=23.050! -[2023-10-15 15:24:55,324][52866] Updated weights for policy 1, policy_version 15370 (0.0008) -[2023-10-15 15:24:55,703][52866] Updated weights for policy 1, policy_version 15380 (0.0007) -[2023-10-15 15:24:56,073][52866] Updated weights for policy 1, policy_version 15390 (0.0009) -[2023-10-15 15:24:58,014][52833] Updated weights for policy 0, policy_version 15330 (0.0008) -[2023-10-15 15:24:58,379][52833] Updated weights for policy 0, policy_version 15340 (0.0007) -[2023-10-15 15:24:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 31457280. Throughput: 0: 1803.9, 1: 1785.8. Samples: 7879640. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:24:58,441][51532] Avg episode reward: [(0, '23.920'), (1, '25.240')] -[2023-10-15 15:24:58,753][52833] Updated weights for policy 0, policy_version 15350 (0.0007) -[2023-10-15 15:24:59,124][52410] Saving new best policy, reward=23.920! -[2023-10-15 15:24:59,126][52833] Updated weights for policy 0, policy_version 15360 (0.0008) -[2023-10-15 15:24:59,641][52866] Updated weights for policy 1, policy_version 15400 (0.0008) -[2023-10-15 15:24:59,997][52866] Updated weights for policy 1, policy_version 15410 (0.0010) -[2023-10-15 15:25:00,374][52866] Updated weights for policy 1, policy_version 15420 (0.0009) -[2023-10-15 15:25:02,867][52833] Updated weights for policy 0, policy_version 15370 (0.0007) -[2023-10-15 15:25:03,242][52833] Updated weights for policy 0, policy_version 15380 (0.0007) -[2023-10-15 15:25:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 31522816. Throughput: 0: 1790.9, 1: 1790.6. Samples: 7889588. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-15 15:25:03,441][51532] Avg episode reward: [(0, '24.980'), (1, '25.030')] -[2023-10-15 15:25:03,606][52833] Updated weights for policy 0, policy_version 15390 (0.0007) -[2023-10-15 15:25:03,678][52410] Saving new best policy, reward=24.980! -[2023-10-15 15:25:04,173][52866] Updated weights for policy 1, policy_version 15430 (0.0010) -[2023-10-15 15:25:04,545][52866] Updated weights for policy 1, policy_version 15440 (0.0008) -[2023-10-15 15:25:04,916][52866] Updated weights for policy 1, policy_version 15450 (0.0010) -[2023-10-15 15:25:07,414][52833] Updated weights for policy 0, policy_version 15400 (0.0007) -[2023-10-15 15:25:07,782][52833] Updated weights for policy 0, policy_version 15410 (0.0008) -[2023-10-15 15:25:08,153][52833] Updated weights for policy 0, policy_version 15420 (0.0009) -[2023-10-15 15:25:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 31621120. Throughput: 0: 1797.4, 1: 1791.8. Samples: 7911806. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-15 15:25:08,442][51532] Avg episode reward: [(0, '24.290'), (1, '23.820')] -[2023-10-15 15:25:08,732][52866] Updated weights for policy 1, policy_version 15460 (0.0008) -[2023-10-15 15:25:09,096][52866] Updated weights for policy 1, policy_version 15470 (0.0009) -[2023-10-15 15:25:09,452][52866] Updated weights for policy 1, policy_version 15480 (0.0007) -[2023-10-15 15:25:11,927][52833] Updated weights for policy 0, policy_version 15430 (0.0008) -[2023-10-15 15:25:12,288][52833] Updated weights for policy 0, policy_version 15440 (0.0009) -[2023-10-15 15:25:12,659][52833] Updated weights for policy 0, policy_version 15450 (0.0008) -[2023-10-15 15:25:13,132][52866] Updated weights for policy 1, policy_version 15490 (0.0009) -[2023-10-15 15:25:13,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 31686656. Throughput: 0: 1792.3, 1: 1804.2. Samples: 7933120. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 15:25:13,442][51532] Avg episode reward: [(0, '23.780'), (1, '23.090')] -[2023-10-15 15:25:13,501][52866] Updated weights for policy 1, policy_version 15500 (0.0011) -[2023-10-15 15:25:13,864][52866] Updated weights for policy 1, policy_version 15510 (0.0010) -[2023-10-15 15:25:14,238][52866] Updated weights for policy 1, policy_version 15520 (0.0011) -[2023-10-15 15:25:16,353][52833] Updated weights for policy 0, policy_version 15460 (0.0010) -[2023-10-15 15:25:16,728][52833] Updated weights for policy 0, policy_version 15470 (0.0008) -[2023-10-15 15:25:17,089][52833] Updated weights for policy 0, policy_version 15480 (0.0009) -[2023-10-15 15:25:18,144][52866] Updated weights for policy 1, policy_version 15530 (0.0008) -[2023-10-15 15:25:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 31752192. Throughput: 0: 1791.6, 1: 1788.4. Samples: 7943926. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 15:25:18,442][51532] Avg episode reward: [(0, '23.870'), (1, '21.990')] -[2023-10-15 15:25:18,510][52866] Updated weights for policy 1, policy_version 15540 (0.0009) -[2023-10-15 15:25:18,880][52866] Updated weights for policy 1, policy_version 15550 (0.0008) -[2023-10-15 15:25:20,781][52833] Updated weights for policy 0, policy_version 15490 (0.0010) -[2023-10-15 15:25:21,157][52833] Updated weights for policy 0, policy_version 15500 (0.0009) -[2023-10-15 15:25:21,520][52833] Updated weights for policy 0, policy_version 15510 (0.0008) -[2023-10-15 15:25:21,881][52833] Updated weights for policy 0, policy_version 15520 (0.0007) -[2023-10-15 15:25:22,498][52866] Updated weights for policy 1, policy_version 15560 (0.0009) -[2023-10-15 15:25:22,854][52866] Updated weights for policy 1, policy_version 15570 (0.0007) -[2023-10-15 15:25:23,226][52866] Updated weights for policy 1, policy_version 15580 (0.0008) -[2023-10-15 15:25:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 31850496. Throughput: 0: 1792.0, 1: 1800.3. Samples: 7965352. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 15:25:23,442][51532] Avg episode reward: [(0, '22.020'), (1, '21.990')] -[2023-10-15 15:25:25,684][52833] Updated weights for policy 0, policy_version 15530 (0.0007) -[2023-10-15 15:25:26,043][52833] Updated weights for policy 0, policy_version 15540 (0.0008) -[2023-10-15 15:25:26,418][52833] Updated weights for policy 0, policy_version 15550 (0.0010) -[2023-10-15 15:25:26,989][52866] Updated weights for policy 1, policy_version 15590 (0.0009) -[2023-10-15 15:25:27,351][52866] Updated weights for policy 1, policy_version 15600 (0.0010) -[2023-10-15 15:25:27,708][52866] Updated weights for policy 1, policy_version 15610 (0.0008) -[2023-10-15 15:25:28,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 31916032. Throughput: 0: 1794.1, 1: 1792.7. Samples: 7986400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:25:28,442][51532] Avg episode reward: [(0, '20.820'), (1, '22.020')] -[2023-10-15 15:25:30,181][52833] Updated weights for policy 0, policy_version 15560 (0.0009) -[2023-10-15 15:25:30,559][52833] Updated weights for policy 0, policy_version 15570 (0.0010) -[2023-10-15 15:25:30,921][52833] Updated weights for policy 0, policy_version 15580 (0.0010) -[2023-10-15 15:25:31,382][52866] Updated weights for policy 1, policy_version 15620 (0.0008) -[2023-10-15 15:25:31,756][52866] Updated weights for policy 1, policy_version 15630 (0.0008) -[2023-10-15 15:25:32,125][52866] Updated weights for policy 1, policy_version 15640 (0.0008) -[2023-10-15 15:25:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31981568. Throughput: 0: 1801.2, 1: 1801.0. Samples: 7998034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:25:33,442][51532] Avg episode reward: [(0, '20.920'), (1, '21.600')] -[2023-10-15 15:25:34,597][52833] Updated weights for policy 0, policy_version 15590 (0.0009) -[2023-10-15 15:25:34,970][52833] Updated weights for policy 0, policy_version 15600 (0.0008) -[2023-10-15 15:25:35,338][52833] Updated weights for policy 0, policy_version 15610 (0.0009) -[2023-10-15 15:25:35,816][52866] Updated weights for policy 1, policy_version 15650 (0.0008) -[2023-10-15 15:25:36,185][52866] Updated weights for policy 1, policy_version 15660 (0.0010) -[2023-10-15 15:25:36,548][52866] Updated weights for policy 1, policy_version 15670 (0.0009) -[2023-10-15 15:25:36,916][52866] Updated weights for policy 1, policy_version 15680 (0.0011) -[2023-10-15 15:25:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32047104. Throughput: 0: 1795.9, 1: 1798.7. Samples: 8018660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:25:38,442][51532] Avg episode reward: [(0, '20.690'), (1, '22.410')] -[2023-10-15 15:25:39,189][52833] Updated weights for policy 0, policy_version 15620 (0.0007) -[2023-10-15 15:25:39,560][52833] Updated weights for policy 0, policy_version 15630 (0.0007) -[2023-10-15 15:25:39,925][52833] Updated weights for policy 0, policy_version 15640 (0.0008) -[2023-10-15 15:25:40,558][52866] Updated weights for policy 1, policy_version 15690 (0.0008) -[2023-10-15 15:25:40,934][52866] Updated weights for policy 1, policy_version 15700 (0.0007) -[2023-10-15 15:25:41,294][52866] Updated weights for policy 1, policy_version 15710 (0.0011) -[2023-10-15 15:25:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 32112640. Throughput: 0: 1790.8, 1: 1796.4. Samples: 8041068. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 15:25:43,442][51532] Avg episode reward: [(0, '21.020'), (1, '22.960')] -[2023-10-15 15:25:43,453][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000015712_16089088.pth... -[2023-10-15 15:25:43,453][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000015648_16023552.pth... -[2023-10-15 15:25:43,484][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000014016_14352384.pth -[2023-10-15 15:25:43,497][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000013984_14319616.pth -[2023-10-15 15:25:43,703][52833] Updated weights for policy 0, policy_version 15650 (0.0008) -[2023-10-15 15:25:44,082][52833] Updated weights for policy 0, policy_version 15660 (0.0011) -[2023-10-15 15:25:44,452][52833] Updated weights for policy 0, policy_version 15670 (0.0010) -[2023-10-15 15:25:44,819][52833] Updated weights for policy 0, policy_version 15680 (0.0010) -[2023-10-15 15:25:45,000][52866] Updated weights for policy 1, policy_version 15720 (0.0009) -[2023-10-15 15:25:45,377][52866] Updated weights for policy 1, policy_version 15730 (0.0007) -[2023-10-15 15:25:45,745][52866] Updated weights for policy 1, policy_version 15740 (0.0007) -[2023-10-15 15:25:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 32178176. Throughput: 0: 1785.2, 1: 1792.6. Samples: 8050590. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 15:25:48,441][51532] Avg episode reward: [(0, '21.530'), (1, '21.670')] -[2023-10-15 15:25:48,627][52833] Updated weights for policy 0, policy_version 15690 (0.0010) -[2023-10-15 15:25:48,991][52833] Updated weights for policy 0, policy_version 15700 (0.0009) -[2023-10-15 15:25:49,359][52833] Updated weights for policy 0, policy_version 15710 (0.0007) -[2023-10-15 15:25:49,484][52866] Updated weights for policy 1, policy_version 15750 (0.0009) -[2023-10-15 15:25:49,857][52866] Updated weights for policy 1, policy_version 15760 (0.0010) -[2023-10-15 15:25:50,224][52866] Updated weights for policy 1, policy_version 15770 (0.0012) -[2023-10-15 15:25:53,167][52833] Updated weights for policy 0, policy_version 15720 (0.0010) -[2023-10-15 15:25:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 32243712. Throughput: 0: 1782.4, 1: 1793.5. Samples: 8072720. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 15:25:53,441][51532] Avg episode reward: [(0, '21.270'), (1, '22.420')] -[2023-10-15 15:25:53,543][52833] Updated weights for policy 0, policy_version 15730 (0.0009) -[2023-10-15 15:25:53,914][52833] Updated weights for policy 0, policy_version 15740 (0.0011) -[2023-10-15 15:25:54,214][52866] Updated weights for policy 1, policy_version 15780 (0.0011) -[2023-10-15 15:25:54,610][52866] Updated weights for policy 1, policy_version 15790 (0.0008) -[2023-10-15 15:25:54,964][52866] Updated weights for policy 1, policy_version 15800 (0.0008) -[2023-10-15 15:25:57,715][52833] Updated weights for policy 0, policy_version 15750 (0.0008) -[2023-10-15 15:25:58,102][52833] Updated weights for policy 0, policy_version 15760 (0.0007) -[2023-10-15 15:25:58,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 32309248. Throughput: 0: 1796.5, 1: 1785.9. Samples: 8094330. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) -[2023-10-15 15:25:58,442][51532] Avg episode reward: [(0, '20.570'), (1, '22.420')] -[2023-10-15 15:25:58,473][52833] Updated weights for policy 0, policy_version 15770 (0.0011) -[2023-10-15 15:25:58,715][52866] Updated weights for policy 1, policy_version 15810 (0.0008) -[2023-10-15 15:25:59,079][52866] Updated weights for policy 1, policy_version 15820 (0.0008) -[2023-10-15 15:25:59,449][52866] Updated weights for policy 1, policy_version 15830 (0.0009) -[2023-10-15 15:25:59,813][52866] Updated weights for policy 1, policy_version 15840 (0.0007) -[2023-10-15 15:26:02,297][52833] Updated weights for policy 0, policy_version 15780 (0.0008) -[2023-10-15 15:26:02,658][52833] Updated weights for policy 0, policy_version 15790 (0.0010) -[2023-10-15 15:26:03,029][52833] Updated weights for policy 0, policy_version 15800 (0.0008) -[2023-10-15 15:26:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 32407552. Throughput: 0: 1776.9, 1: 1788.9. Samples: 8104386. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) -[2023-10-15 15:26:03,441][51532] Avg episode reward: [(0, '20.780'), (1, '24.120')] -[2023-10-15 15:26:03,569][52866] Updated weights for policy 1, policy_version 15850 (0.0007) -[2023-10-15 15:26:03,945][52866] Updated weights for policy 1, policy_version 15860 (0.0007) -[2023-10-15 15:26:04,309][52866] Updated weights for policy 1, policy_version 15870 (0.0008) -[2023-10-15 15:26:06,780][52833] Updated weights for policy 0, policy_version 15810 (0.0008) -[2023-10-15 15:26:07,149][52833] Updated weights for policy 0, policy_version 15820 (0.0011) -[2023-10-15 15:26:07,524][52833] Updated weights for policy 0, policy_version 15830 (0.0008) -[2023-10-15 15:26:07,900][52833] Updated weights for policy 0, policy_version 15840 (0.0008) -[2023-10-15 15:26:08,024][52866] Updated weights for policy 1, policy_version 15880 (0.0008) -[2023-10-15 15:26:08,392][52866] Updated weights for policy 1, policy_version 15890 (0.0010) -[2023-10-15 15:26:08,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 32473088. Throughput: 0: 1803.6, 1: 1789.5. Samples: 8127042. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 15:26:08,441][51532] Avg episode reward: [(0, '21.800'), (1, '24.240')] -[2023-10-15 15:26:08,758][52866] Updated weights for policy 1, policy_version 15900 (0.0009) -[2023-10-15 15:26:11,736][52833] Updated weights for policy 0, policy_version 15850 (0.0011) -[2023-10-15 15:26:12,109][52833] Updated weights for policy 0, policy_version 15860 (0.0011) -[2023-10-15 15:26:12,480][52833] Updated weights for policy 0, policy_version 15870 (0.0009) -[2023-10-15 15:26:12,543][52866] Updated weights for policy 1, policy_version 15910 (0.0009) -[2023-10-15 15:26:12,904][52866] Updated weights for policy 1, policy_version 15920 (0.0010) -[2023-10-15 15:26:13,275][52866] Updated weights for policy 1, policy_version 15930 (0.0010) -[2023-10-15 15:26:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 32538624. Throughput: 0: 1768.9, 1: 1803.1. Samples: 8147138. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 15:26:13,441][51532] Avg episode reward: [(0, '19.400'), (1, '23.540')] -[2023-10-15 15:26:16,053][52833] Updated weights for policy 0, policy_version 15880 (0.0007) -[2023-10-15 15:26:16,418][52833] Updated weights for policy 0, policy_version 15890 (0.0008) -[2023-10-15 15:26:16,788][52833] Updated weights for policy 0, policy_version 15900 (0.0008) -[2023-10-15 15:26:17,036][52866] Updated weights for policy 1, policy_version 15940 (0.0009) -[2023-10-15 15:26:17,409][52866] Updated weights for policy 1, policy_version 15950 (0.0008) -[2023-10-15 15:26:17,776][52866] Updated weights for policy 1, policy_version 15960 (0.0008) -[2023-10-15 15:26:18,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 32636928. Throughput: 0: 1795.7, 1: 1787.9. Samples: 8159296. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 15:26:18,441][51532] Avg episode reward: [(0, '22.140'), (1, '24.750')] -[2023-10-15 15:26:20,634][52833] Updated weights for policy 0, policy_version 15910 (0.0009) -[2023-10-15 15:26:21,007][52833] Updated weights for policy 0, policy_version 15920 (0.0008) -[2023-10-15 15:26:21,370][52833] Updated weights for policy 0, policy_version 15930 (0.0010) -[2023-10-15 15:26:21,445][52866] Updated weights for policy 1, policy_version 15970 (0.0008) -[2023-10-15 15:26:21,814][52866] Updated weights for policy 1, policy_version 15980 (0.0008) -[2023-10-15 15:26:22,185][52866] Updated weights for policy 1, policy_version 15990 (0.0007) -[2023-10-15 15:26:22,546][52866] Updated weights for policy 1, policy_version 16000 (0.0007) -[2023-10-15 15:26:23,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32702464. Throughput: 0: 1767.4, 1: 1806.1. Samples: 8179468. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-15 15:26:23,442][51532] Avg episode reward: [(0, '21.250'), (1, '25.080')] -[2023-10-15 15:26:25,101][52833] Updated weights for policy 0, policy_version 15940 (0.0009) -[2023-10-15 15:26:25,465][52833] Updated weights for policy 0, policy_version 15950 (0.0008) -[2023-10-15 15:26:25,826][52833] Updated weights for policy 0, policy_version 15960 (0.0009) -[2023-10-15 15:26:26,439][52866] Updated weights for policy 1, policy_version 16010 (0.0007) -[2023-10-15 15:26:26,794][52866] Updated weights for policy 1, policy_version 16020 (0.0008) -[2023-10-15 15:26:27,162][52866] Updated weights for policy 1, policy_version 16030 (0.0008) -[2023-10-15 15:26:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32768000. Throughput: 0: 1764.6, 1: 1788.5. Samples: 8200960. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-15 15:26:28,442][51532] Avg episode reward: [(0, '21.950'), (1, '22.220')] -[2023-10-15 15:26:29,601][52833] Updated weights for policy 0, policy_version 15970 (0.0009) -[2023-10-15 15:26:29,962][52833] Updated weights for policy 0, policy_version 15980 (0.0008) -[2023-10-15 15:26:30,334][52833] Updated weights for policy 0, policy_version 15990 (0.0008) -[2023-10-15 15:26:30,698][52833] Updated weights for policy 0, policy_version 16000 (0.0009) -[2023-10-15 15:26:30,864][52866] Updated weights for policy 1, policy_version 16040 (0.0008) -[2023-10-15 15:26:31,236][52866] Updated weights for policy 1, policy_version 16050 (0.0008) -[2023-10-15 15:26:31,610][52866] Updated weights for policy 1, policy_version 16060 (0.0008) -[2023-10-15 15:26:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32833536. Throughput: 0: 1768.8, 1: 1813.5. Samples: 8211794. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-15 15:26:33,441][51532] Avg episode reward: [(0, '20.490'), (1, '21.730')] -[2023-10-15 15:26:34,597][52833] Updated weights for policy 0, policy_version 16010 (0.0008) -[2023-10-15 15:26:34,970][52833] Updated weights for policy 0, policy_version 16020 (0.0010) -[2023-10-15 15:26:35,294][52866] Updated weights for policy 1, policy_version 16070 (0.0009) -[2023-10-15 15:26:35,336][52833] Updated weights for policy 0, policy_version 16030 (0.0009) -[2023-10-15 15:26:35,667][52866] Updated weights for policy 1, policy_version 16080 (0.0009) -[2023-10-15 15:26:36,035][52866] Updated weights for policy 1, policy_version 16090 (0.0008) -[2023-10-15 15:26:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 32899072. Throughput: 0: 1776.4, 1: 1797.6. Samples: 8233550. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:26:38,442][51532] Avg episode reward: [(0, '21.310'), (1, '23.010')] -[2023-10-15 15:26:39,091][52833] Updated weights for policy 0, policy_version 16040 (0.0011) -[2023-10-15 15:26:39,457][52833] Updated weights for policy 0, policy_version 16050 (0.0010) -[2023-10-15 15:26:39,772][52866] Updated weights for policy 1, policy_version 16100 (0.0009) -[2023-10-15 15:26:39,830][52833] Updated weights for policy 0, policy_version 16060 (0.0007) -[2023-10-15 15:26:40,157][52866] Updated weights for policy 1, policy_version 16110 (0.0009) -[2023-10-15 15:26:40,522][52866] Updated weights for policy 1, policy_version 16120 (0.0010) -[2023-10-15 15:26:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 32964608. Throughput: 0: 1791.2, 1: 1799.9. Samples: 8255926. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:26:43,441][51532] Avg episode reward: [(0, '19.150'), (1, '24.190')] -[2023-10-15 15:26:43,643][52833] Updated weights for policy 0, policy_version 16070 (0.0009) -[2023-10-15 15:26:44,023][52833] Updated weights for policy 0, policy_version 16080 (0.0008) -[2023-10-15 15:26:44,279][52866] Updated weights for policy 1, policy_version 16130 (0.0010) -[2023-10-15 15:26:44,394][52833] Updated weights for policy 0, policy_version 16090 (0.0007) -[2023-10-15 15:26:44,646][52866] Updated weights for policy 1, policy_version 16140 (0.0010) -[2023-10-15 15:26:45,018][52866] Updated weights for policy 1, policy_version 16150 (0.0010) -[2023-10-15 15:26:45,380][52866] Updated weights for policy 1, policy_version 16160 (0.0008) -[2023-10-15 15:26:48,001][52833] Updated weights for policy 0, policy_version 16100 (0.0008) -[2023-10-15 15:26:48,377][52833] Updated weights for policy 0, policy_version 16110 (0.0008) -[2023-10-15 15:26:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 33030144. Throughput: 0: 1782.7, 1: 1800.4. Samples: 8265624. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:26:48,441][51532] Avg episode reward: [(0, '20.900'), (1, '22.880')] -[2023-10-15 15:26:48,742][52833] Updated weights for policy 0, policy_version 16120 (0.0009) -[2023-10-15 15:26:49,226][52866] Updated weights for policy 1, policy_version 16170 (0.0007) -[2023-10-15 15:26:49,589][52866] Updated weights for policy 1, policy_version 16180 (0.0007) -[2023-10-15 15:26:49,963][52866] Updated weights for policy 1, policy_version 16190 (0.0008) -[2023-10-15 15:26:52,575][52833] Updated weights for policy 0, policy_version 16130 (0.0008) -[2023-10-15 15:26:52,945][52833] Updated weights for policy 0, policy_version 16140 (0.0007) -[2023-10-15 15:26:53,312][52833] Updated weights for policy 0, policy_version 16150 (0.0008) -[2023-10-15 15:26:53,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 33095680. Throughput: 0: 1783.5, 1: 1790.2. Samples: 8287860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:26:53,442][51532] Avg episode reward: [(0, '20.780'), (1, '21.650')] -[2023-10-15 15:26:53,678][52833] Updated weights for policy 0, policy_version 16160 (0.0008) -[2023-10-15 15:26:53,706][52866] Updated weights for policy 1, policy_version 16200 (0.0009) -[2023-10-15 15:26:54,068][52866] Updated weights for policy 1, policy_version 16210 (0.0008) -[2023-10-15 15:26:54,438][52866] Updated weights for policy 1, policy_version 16220 (0.0009) -[2023-10-15 15:26:57,500][52833] Updated weights for policy 0, policy_version 16170 (0.0008) -[2023-10-15 15:26:57,880][52833] Updated weights for policy 0, policy_version 16180 (0.0009) -[2023-10-15 15:26:58,240][52833] Updated weights for policy 0, policy_version 16190 (0.0008) -[2023-10-15 15:26:58,274][52866] Updated weights for policy 1, policy_version 16230 (0.0008) -[2023-10-15 15:26:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 33193984. Throughput: 0: 1795.2, 1: 1813.1. Samples: 8309510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:26:58,441][51532] Avg episode reward: [(0, '21.400'), (1, '23.980')] -[2023-10-15 15:26:58,635][52866] Updated weights for policy 1, policy_version 16240 (0.0009) -[2023-10-15 15:26:59,008][52866] Updated weights for policy 1, policy_version 16250 (0.0007) -[2023-10-15 15:27:01,912][52833] Updated weights for policy 0, policy_version 16200 (0.0008) -[2023-10-15 15:27:02,284][52833] Updated weights for policy 0, policy_version 16210 (0.0009) -[2023-10-15 15:27:02,600][52866] Updated weights for policy 1, policy_version 16260 (0.0008) -[2023-10-15 15:27:02,652][52833] Updated weights for policy 0, policy_version 16220 (0.0007) -[2023-10-15 15:27:02,970][52866] Updated weights for policy 1, policy_version 16270 (0.0008) -[2023-10-15 15:27:03,333][52866] Updated weights for policy 1, policy_version 16280 (0.0007) -[2023-10-15 15:27:03,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 33259520. Throughput: 0: 1787.1, 1: 1793.7. Samples: 8320436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:03,442][51532] Avg episode reward: [(0, '22.180'), (1, '22.970')] -[2023-10-15 15:27:06,558][52833] Updated weights for policy 0, policy_version 16230 (0.0008) -[2023-10-15 15:27:06,929][52833] Updated weights for policy 0, policy_version 16240 (0.0011) -[2023-10-15 15:27:07,251][52866] Updated weights for policy 1, policy_version 16290 (0.0008) -[2023-10-15 15:27:07,293][52833] Updated weights for policy 0, policy_version 16250 (0.0008) -[2023-10-15 15:27:07,616][52866] Updated weights for policy 1, policy_version 16300 (0.0008) -[2023-10-15 15:27:07,980][52866] Updated weights for policy 1, policy_version 16310 (0.0008) -[2023-10-15 15:27:08,344][52866] Updated weights for policy 1, policy_version 16320 (0.0008) -[2023-10-15 15:27:08,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 33357824. Throughput: 0: 1798.6, 1: 1809.9. Samples: 8341854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:08,442][51532] Avg episode reward: [(0, '23.190'), (1, '22.750')] -[2023-10-15 15:27:10,959][52833] Updated weights for policy 0, policy_version 16260 (0.0010) -[2023-10-15 15:27:11,335][52833] Updated weights for policy 0, policy_version 16270 (0.0008) -[2023-10-15 15:27:11,697][52833] Updated weights for policy 0, policy_version 16280 (0.0007) -[2023-10-15 15:27:12,082][52866] Updated weights for policy 1, policy_version 16330 (0.0007) -[2023-10-15 15:27:12,443][52866] Updated weights for policy 1, policy_version 16340 (0.0009) -[2023-10-15 15:27:12,802][52866] Updated weights for policy 1, policy_version 16350 (0.0008) -[2023-10-15 15:27:13,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 33423360. Throughput: 0: 1788.4, 1: 1791.8. Samples: 8362070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:13,441][51532] Avg episode reward: [(0, '21.490'), (1, '22.760')] -[2023-10-15 15:27:15,417][52833] Updated weights for policy 0, policy_version 16290 (0.0009) -[2023-10-15 15:27:15,793][52833] Updated weights for policy 0, policy_version 16300 (0.0009) -[2023-10-15 15:27:16,158][52833] Updated weights for policy 0, policy_version 16310 (0.0009) -[2023-10-15 15:27:16,522][52833] Updated weights for policy 0, policy_version 16320 (0.0008) -[2023-10-15 15:27:16,585][52866] Updated weights for policy 1, policy_version 16360 (0.0008) -[2023-10-15 15:27:16,961][52866] Updated weights for policy 1, policy_version 16370 (0.0010) -[2023-10-15 15:27:17,332][52866] Updated weights for policy 1, policy_version 16380 (0.0007) -[2023-10-15 15:27:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 33488896. Throughput: 0: 1805.0, 1: 1801.6. Samples: 8374094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:18,442][51532] Avg episode reward: [(0, '22.560'), (1, '23.970')] -[2023-10-15 15:27:20,350][52833] Updated weights for policy 0, policy_version 16330 (0.0010) -[2023-10-15 15:27:20,728][52833] Updated weights for policy 0, policy_version 16340 (0.0007) -[2023-10-15 15:27:21,088][52833] Updated weights for policy 0, policy_version 16350 (0.0009) -[2023-10-15 15:27:21,181][52866] Updated weights for policy 1, policy_version 16390 (0.0008) -[2023-10-15 15:27:21,545][52866] Updated weights for policy 1, policy_version 16400 (0.0010) -[2023-10-15 15:27:21,908][52866] Updated weights for policy 1, policy_version 16410 (0.0009) -[2023-10-15 15:27:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 33554432. Throughput: 0: 1786.5, 1: 1783.0. Samples: 8394178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:23,442][51532] Avg episode reward: [(0, '20.810'), (1, '23.820')] -[2023-10-15 15:27:24,993][52833] Updated weights for policy 0, policy_version 16360 (0.0009) -[2023-10-15 15:27:25,363][52833] Updated weights for policy 0, policy_version 16370 (0.0011) -[2023-10-15 15:27:25,663][52866] Updated weights for policy 1, policy_version 16420 (0.0008) -[2023-10-15 15:27:25,728][52833] Updated weights for policy 0, policy_version 16380 (0.0007) -[2023-10-15 15:27:26,048][52866] Updated weights for policy 1, policy_version 16430 (0.0008) -[2023-10-15 15:27:26,419][52866] Updated weights for policy 1, policy_version 16440 (0.0010) -[2023-10-15 15:27:28,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 33619968. Throughput: 0: 1781.7, 1: 1783.6. Samples: 8416362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:28,441][51532] Avg episode reward: [(0, '19.970'), (1, '23.240')] -[2023-10-15 15:27:29,669][52833] Updated weights for policy 0, policy_version 16390 (0.0009) -[2023-10-15 15:27:30,052][52833] Updated weights for policy 0, policy_version 16400 (0.0007) -[2023-10-15 15:27:30,144][52866] Updated weights for policy 1, policy_version 16450 (0.0008) -[2023-10-15 15:27:30,425][52833] Updated weights for policy 0, policy_version 16410 (0.0009) -[2023-10-15 15:27:30,509][52866] Updated weights for policy 1, policy_version 16460 (0.0009) -[2023-10-15 15:27:30,872][52866] Updated weights for policy 1, policy_version 16470 (0.0008) -[2023-10-15 15:27:31,239][52866] Updated weights for policy 1, policy_version 16480 (0.0009) -[2023-10-15 15:27:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 33685504. Throughput: 0: 1779.7, 1: 1793.7. Samples: 8426428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:33,441][51532] Avg episode reward: [(0, '19.390'), (1, '23.540')] -[2023-10-15 15:27:34,070][52833] Updated weights for policy 0, policy_version 16420 (0.0008) -[2023-10-15 15:27:34,442][52833] Updated weights for policy 0, policy_version 16430 (0.0007) -[2023-10-15 15:27:34,816][52833] Updated weights for policy 0, policy_version 16440 (0.0008) -[2023-10-15 15:27:35,043][52866] Updated weights for policy 1, policy_version 16490 (0.0008) -[2023-10-15 15:27:35,402][52866] Updated weights for policy 1, policy_version 16500 (0.0007) -[2023-10-15 15:27:35,773][52866] Updated weights for policy 1, policy_version 16510 (0.0008) -[2023-10-15 15:27:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 33751040. Throughput: 0: 1774.7, 1: 1787.1. Samples: 8448140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:38,441][51532] Avg episode reward: [(0, '20.090'), (1, '25.260')] -[2023-10-15 15:27:38,655][52833] Updated weights for policy 0, policy_version 16450 (0.0009) -[2023-10-15 15:27:39,022][52833] Updated weights for policy 0, policy_version 16460 (0.0007) -[2023-10-15 15:27:39,394][52833] Updated weights for policy 0, policy_version 16470 (0.0008) -[2023-10-15 15:27:39,630][52866] Updated weights for policy 1, policy_version 16520 (0.0008) -[2023-10-15 15:27:39,754][52833] Updated weights for policy 0, policy_version 16480 (0.0007) -[2023-10-15 15:27:39,996][52866] Updated weights for policy 1, policy_version 16530 (0.0011) -[2023-10-15 15:27:40,366][52866] Updated weights for policy 1, policy_version 16540 (0.0008) -[2023-10-15 15:27:43,441][52833] Updated weights for policy 0, policy_version 16490 (0.0007) -[2023-10-15 15:27:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 33816576. Throughput: 0: 1798.5, 1: 1779.5. Samples: 8470522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:43,442][51532] Avg episode reward: [(0, '20.290'), (1, '24.330')] -[2023-10-15 15:27:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000016544_16941056.pth... -[2023-10-15 15:27:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000014880_15237120.pth -[2023-10-15 15:27:43,492][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000016544_16941056.pth -[2023-10-15 15:27:43,813][52833] Updated weights for policy 0, policy_version 16500 (0.0007) -[2023-10-15 15:27:44,082][52866] Updated weights for policy 1, policy_version 16550 (0.0008) -[2023-10-15 15:27:44,184][52833] Updated weights for policy 0, policy_version 16510 (0.0008) -[2023-10-15 15:27:44,259][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000016512_16908288.pth... -[2023-10-15 15:27:44,293][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000014816_15171584.pth -[2023-10-15 15:27:44,297][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000016512_16908288.pth -[2023-10-15 15:27:44,445][52866] Updated weights for policy 1, policy_version 16560 (0.0009) -[2023-10-15 15:27:44,808][52866] Updated weights for policy 1, policy_version 16570 (0.0007) -[2023-10-15 15:27:48,037][52833] Updated weights for policy 0, policy_version 16520 (0.0009) -[2023-10-15 15:27:48,398][52833] Updated weights for policy 0, policy_version 16530 (0.0008) -[2023-10-15 15:27:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 33882112. Throughput: 0: 1776.1, 1: 1776.3. Samples: 8480294. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-15 15:27:48,441][51532] Avg episode reward: [(0, '21.710'), (1, '24.100')] -[2023-10-15 15:27:48,576][52866] Updated weights for policy 1, policy_version 16580 (0.0008) -[2023-10-15 15:27:48,764][52833] Updated weights for policy 0, policy_version 16540 (0.0010) -[2023-10-15 15:27:48,945][52866] Updated weights for policy 1, policy_version 16590 (0.0010) -[2023-10-15 15:27:49,314][52866] Updated weights for policy 1, policy_version 16600 (0.0010) -[2023-10-15 15:27:52,488][52833] Updated weights for policy 0, policy_version 16550 (0.0009) -[2023-10-15 15:27:52,865][52833] Updated weights for policy 0, policy_version 16560 (0.0008) -[2023-10-15 15:27:52,943][52866] Updated weights for policy 1, policy_version 16610 (0.0009) -[2023-10-15 15:27:53,235][52833] Updated weights for policy 0, policy_version 16570 (0.0009) -[2023-10-15 15:27:53,303][52866] Updated weights for policy 1, policy_version 16620 (0.0007) -[2023-10-15 15:27:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 33947648. Throughput: 0: 1800.0, 1: 1780.8. Samples: 8502990. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-15 15:27:53,442][51532] Avg episode reward: [(0, '21.970'), (1, '24.880')] -[2023-10-15 15:27:53,674][52866] Updated weights for policy 1, policy_version 16630 (0.0008) -[2023-10-15 15:27:54,033][52866] Updated weights for policy 1, policy_version 16640 (0.0007) -[2023-10-15 15:27:56,897][52833] Updated weights for policy 0, policy_version 16580 (0.0009) -[2023-10-15 15:27:57,273][52833] Updated weights for policy 0, policy_version 16590 (0.0010) -[2023-10-15 15:27:57,645][52833] Updated weights for policy 0, policy_version 16600 (0.0009) -[2023-10-15 15:27:57,895][52866] Updated weights for policy 1, policy_version 16650 (0.0008) -[2023-10-15 15:27:58,263][52866] Updated weights for policy 1, policy_version 16660 (0.0009) -[2023-10-15 15:27:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 34045952. Throughput: 0: 1784.9, 1: 1804.8. Samples: 8523604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:27:58,441][51532] Avg episode reward: [(0, '22.870'), (1, '24.900')] -[2023-10-15 15:27:58,635][52866] Updated weights for policy 1, policy_version 16670 (0.0010) -[2023-10-15 15:28:01,287][52833] Updated weights for policy 0, policy_version 16610 (0.0008) -[2023-10-15 15:28:01,657][52833] Updated weights for policy 0, policy_version 16620 (0.0008) -[2023-10-15 15:28:02,025][52833] Updated weights for policy 0, policy_version 16630 (0.0007) -[2023-10-15 15:28:02,392][52833] Updated weights for policy 0, policy_version 16640 (0.0008) -[2023-10-15 15:28:02,530][52866] Updated weights for policy 1, policy_version 16680 (0.0008) -[2023-10-15 15:28:02,908][52866] Updated weights for policy 1, policy_version 16690 (0.0008) -[2023-10-15 15:28:03,279][52866] Updated weights for policy 1, policy_version 16700 (0.0008) -[2023-10-15 15:28:03,441][51532] Fps is (10 sec: 19661.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 34144256. Throughput: 0: 1800.5, 1: 1777.7. Samples: 8535112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:28:03,441][51532] Avg episode reward: [(0, '23.780'), (1, '25.040')] -[2023-10-15 15:28:06,132][52833] Updated weights for policy 0, policy_version 16650 (0.0011) -[2023-10-15 15:28:06,499][52833] Updated weights for policy 0, policy_version 16660 (0.0008) -[2023-10-15 15:28:06,870][52833] Updated weights for policy 0, policy_version 16670 (0.0007) -[2023-10-15 15:28:07,022][52866] Updated weights for policy 1, policy_version 16710 (0.0010) -[2023-10-15 15:28:07,392][52866] Updated weights for policy 1, policy_version 16720 (0.0007) -[2023-10-15 15:28:07,763][52866] Updated weights for policy 1, policy_version 16730 (0.0008) -[2023-10-15 15:28:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34209792. Throughput: 0: 1790.2, 1: 1807.8. Samples: 8556088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:28:08,442][51532] Avg episode reward: [(0, '23.970'), (1, '23.230')] -[2023-10-15 15:28:10,558][52833] Updated weights for policy 0, policy_version 16680 (0.0009) -[2023-10-15 15:28:10,930][52833] Updated weights for policy 0, policy_version 16690 (0.0009) -[2023-10-15 15:28:11,302][52833] Updated weights for policy 0, policy_version 16700 (0.0009) -[2023-10-15 15:28:11,488][52866] Updated weights for policy 1, policy_version 16740 (0.0011) -[2023-10-15 15:28:11,891][52866] Updated weights for policy 1, policy_version 16750 (0.0009) -[2023-10-15 15:28:12,256][52866] Updated weights for policy 1, policy_version 16760 (0.0008) -[2023-10-15 15:28:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34275328. Throughput: 0: 1789.8, 1: 1779.5. Samples: 8576982. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 15:28:13,441][51532] Avg episode reward: [(0, '23.310'), (1, '22.700')] -[2023-10-15 15:28:15,171][52833] Updated weights for policy 0, policy_version 16710 (0.0008) -[2023-10-15 15:28:15,556][52833] Updated weights for policy 0, policy_version 16720 (0.0009) -[2023-10-15 15:28:15,931][52833] Updated weights for policy 0, policy_version 16730 (0.0008) -[2023-10-15 15:28:15,974][52866] Updated weights for policy 1, policy_version 16770 (0.0007) -[2023-10-15 15:28:16,342][52866] Updated weights for policy 1, policy_version 16780 (0.0008) -[2023-10-15 15:28:16,706][52866] Updated weights for policy 1, policy_version 16790 (0.0009) -[2023-10-15 15:28:17,079][52866] Updated weights for policy 1, policy_version 16800 (0.0007) -[2023-10-15 15:28:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 34340864. Throughput: 0: 1798.8, 1: 1800.0. Samples: 8588374. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 15:28:18,441][51532] Avg episode reward: [(0, '23.890'), (1, '22.080')] -[2023-10-15 15:28:19,765][52833] Updated weights for policy 0, policy_version 16740 (0.0008) -[2023-10-15 15:28:20,126][52833] Updated weights for policy 0, policy_version 16750 (0.0011) -[2023-10-15 15:28:20,490][52833] Updated weights for policy 0, policy_version 16760 (0.0010) -[2023-10-15 15:28:20,733][52866] Updated weights for policy 1, policy_version 16810 (0.0007) -[2023-10-15 15:28:21,094][52866] Updated weights for policy 1, policy_version 16820 (0.0009) -[2023-10-15 15:28:21,457][52866] Updated weights for policy 1, policy_version 16830 (0.0009) -[2023-10-15 15:28:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 34406400. Throughput: 0: 1792.5, 1: 1785.9. Samples: 8609168. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 15:28:23,442][51532] Avg episode reward: [(0, '22.310'), (1, '19.760')] -[2023-10-15 15:28:24,233][52833] Updated weights for policy 0, policy_version 16770 (0.0009) -[2023-10-15 15:28:24,605][52833] Updated weights for policy 0, policy_version 16780 (0.0008) -[2023-10-15 15:28:24,934][52866] Updated weights for policy 1, policy_version 16840 (0.0008) -[2023-10-15 15:28:24,974][52833] Updated weights for policy 0, policy_version 16790 (0.0009) -[2023-10-15 15:28:25,298][52866] Updated weights for policy 1, policy_version 16850 (0.0007) -[2023-10-15 15:28:25,339][52833] Updated weights for policy 0, policy_version 16800 (0.0008) -[2023-10-15 15:28:25,668][52866] Updated weights for policy 1, policy_version 16860 (0.0007) -[2023-10-15 15:28:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 34471936. Throughput: 0: 1787.0, 1: 1795.3. Samples: 8631722. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:28:28,442][51532] Avg episode reward: [(0, '22.370'), (1, '19.400')] -[2023-10-15 15:28:29,120][52833] Updated weights for policy 0, policy_version 16810 (0.0007) -[2023-10-15 15:28:29,423][52866] Updated weights for policy 1, policy_version 16870 (0.0007) -[2023-10-15 15:28:29,495][52833] Updated weights for policy 0, policy_version 16820 (0.0007) -[2023-10-15 15:28:29,789][52866] Updated weights for policy 1, policy_version 16880 (0.0009) -[2023-10-15 15:28:29,866][52833] Updated weights for policy 0, policy_version 16830 (0.0007) -[2023-10-15 15:28:30,151][52866] Updated weights for policy 1, policy_version 16890 (0.0008) -[2023-10-15 15:28:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 34537472. Throughput: 0: 1787.1, 1: 1796.8. Samples: 8641566. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:28:33,441][51532] Avg episode reward: [(0, '22.260'), (1, '19.600')] -[2023-10-15 15:28:33,607][52833] Updated weights for policy 0, policy_version 16840 (0.0008) -[2023-10-15 15:28:33,882][52866] Updated weights for policy 1, policy_version 16900 (0.0008) -[2023-10-15 15:28:33,976][52833] Updated weights for policy 0, policy_version 16850 (0.0011) -[2023-10-15 15:28:34,252][52866] Updated weights for policy 1, policy_version 16910 (0.0007) -[2023-10-15 15:28:34,348][52833] Updated weights for policy 0, policy_version 16860 (0.0008) -[2023-10-15 15:28:34,615][52866] Updated weights for policy 1, policy_version 16920 (0.0008) -[2023-10-15 15:28:38,119][52833] Updated weights for policy 0, policy_version 16870 (0.0011) -[2023-10-15 15:28:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 34603008. Throughput: 0: 1784.0, 1: 1791.5. Samples: 8663886. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:28:38,441][51532] Avg episode reward: [(0, '22.350'), (1, '20.490')] -[2023-10-15 15:28:38,463][52866] Updated weights for policy 1, policy_version 16930 (0.0008) -[2023-10-15 15:28:38,486][52833] Updated weights for policy 0, policy_version 16880 (0.0008) -[2023-10-15 15:28:38,831][52866] Updated weights for policy 1, policy_version 16940 (0.0007) -[2023-10-15 15:28:38,852][52833] Updated weights for policy 0, policy_version 16890 (0.0008) -[2023-10-15 15:28:39,200][52866] Updated weights for policy 1, policy_version 16950 (0.0007) -[2023-10-15 15:28:39,574][52866] Updated weights for policy 1, policy_version 16960 (0.0009) -[2023-10-15 15:28:42,656][52833] Updated weights for policy 0, policy_version 16900 (0.0008) -[2023-10-15 15:28:43,025][52833] Updated weights for policy 0, policy_version 16910 (0.0008) -[2023-10-15 15:28:43,260][52866] Updated weights for policy 1, policy_version 16970 (0.0007) -[2023-10-15 15:28:43,390][52833] Updated weights for policy 0, policy_version 16920 (0.0008) -[2023-10-15 15:28:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14218.0). Total num frames: 34668544. Throughput: 0: 1797.3, 1: 1806.3. Samples: 8685766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:28:43,441][51532] Avg episode reward: [(0, '23.930'), (1, '21.170')] -[2023-10-15 15:28:43,632][52866] Updated weights for policy 1, policy_version 16980 (0.0008) -[2023-10-15 15:28:44,005][52866] Updated weights for policy 1, policy_version 16990 (0.0008) -[2023-10-15 15:28:47,336][52833] Updated weights for policy 0, policy_version 16930 (0.0008) -[2023-10-15 15:28:47,708][52833] Updated weights for policy 0, policy_version 16940 (0.0009) -[2023-10-15 15:28:47,761][52866] Updated weights for policy 1, policy_version 17000 (0.0009) -[2023-10-15 15:28:48,076][52833] Updated weights for policy 0, policy_version 16950 (0.0008) -[2023-10-15 15:28:48,133][52866] Updated weights for policy 1, policy_version 17010 (0.0008) -[2023-10-15 15:28:48,434][52833] Updated weights for policy 0, policy_version 16960 (0.0009) -[2023-10-15 15:28:48,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 34766848. Throughput: 0: 1772.6, 1: 1800.2. Samples: 8695886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:28:48,442][51532] Avg episode reward: [(0, '23.930'), (1, '21.500')] -[2023-10-15 15:28:48,493][52866] Updated weights for policy 1, policy_version 17020 (0.0010) -[2023-10-15 15:28:52,287][52833] Updated weights for policy 0, policy_version 16970 (0.0007) -[2023-10-15 15:28:52,369][52866] Updated weights for policy 1, policy_version 17030 (0.0008) -[2023-10-15 15:28:52,651][52833] Updated weights for policy 0, policy_version 16980 (0.0009) -[2023-10-15 15:28:52,735][52866] Updated weights for policy 1, policy_version 17040 (0.0008) -[2023-10-15 15:28:53,025][52833] Updated weights for policy 0, policy_version 16990 (0.0009) -[2023-10-15 15:28:53,103][52866] Updated weights for policy 1, policy_version 17050 (0.0009) -[2023-10-15 15:28:53,441][51532] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 34865152. Throughput: 0: 1795.4, 1: 1802.9. Samples: 8718012. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-15 15:28:53,442][51532] Avg episode reward: [(0, '24.880'), (1, '21.880')] -[2023-10-15 15:28:56,701][52833] Updated weights for policy 0, policy_version 17000 (0.0009) -[2023-10-15 15:28:57,039][52866] Updated weights for policy 1, policy_version 17060 (0.0007) -[2023-10-15 15:28:57,066][52833] Updated weights for policy 0, policy_version 17010 (0.0008) -[2023-10-15 15:28:57,433][52866] Updated weights for policy 1, policy_version 17070 (0.0007) -[2023-10-15 15:28:57,435][52833] Updated weights for policy 0, policy_version 17020 (0.0008) -[2023-10-15 15:28:57,806][52866] Updated weights for policy 1, policy_version 17080 (0.0009) -[2023-10-15 15:28:58,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 34930688. Throughput: 0: 1765.2, 1: 1799.7. Samples: 8737404. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-15 15:28:58,442][51532] Avg episode reward: [(0, '25.700'), (1, '22.320')] -[2023-10-15 15:28:58,449][52410] Saving new best policy, reward=25.700! -[2023-10-15 15:29:01,015][52833] Updated weights for policy 0, policy_version 17030 (0.0008) -[2023-10-15 15:29:01,385][52833] Updated weights for policy 0, policy_version 17040 (0.0008) -[2023-10-15 15:29:01,505][52866] Updated weights for policy 1, policy_version 17090 (0.0008) -[2023-10-15 15:29:01,759][52833] Updated weights for policy 0, policy_version 17050 (0.0008) -[2023-10-15 15:29:01,884][52866] Updated weights for policy 1, policy_version 17100 (0.0007) -[2023-10-15 15:29:02,253][52866] Updated weights for policy 1, policy_version 17110 (0.0009) -[2023-10-15 15:29:02,609][52866] Updated weights for policy 1, policy_version 17120 (0.0012) -[2023-10-15 15:29:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 34996224. Throughput: 0: 1797.8, 1: 1797.5. Samples: 8750164. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-15 15:29:03,442][51532] Avg episode reward: [(0, '25.660'), (1, '22.730')] -[2023-10-15 15:29:05,594][52833] Updated weights for policy 0, policy_version 17060 (0.0007) -[2023-10-15 15:29:05,967][52833] Updated weights for policy 0, policy_version 17070 (0.0007) -[2023-10-15 15:29:06,324][52833] Updated weights for policy 0, policy_version 17080 (0.0007) -[2023-10-15 15:29:06,389][52866] Updated weights for policy 1, policy_version 17130 (0.0008) -[2023-10-15 15:29:06,762][52866] Updated weights for policy 1, policy_version 17140 (0.0009) -[2023-10-15 15:29:07,123][52866] Updated weights for policy 1, policy_version 17150 (0.0010) -[2023-10-15 15:29:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35061760. Throughput: 0: 1775.3, 1: 1794.8. Samples: 8769822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:29:08,441][51532] Avg episode reward: [(0, '26.540'), (1, '22.560')] -[2023-10-15 15:29:08,442][52410] Saving new best policy, reward=26.540! -[2023-10-15 15:29:10,060][52833] Updated weights for policy 0, policy_version 17090 (0.0008) -[2023-10-15 15:29:10,430][52833] Updated weights for policy 0, policy_version 17100 (0.0007) -[2023-10-15 15:29:10,790][52833] Updated weights for policy 0, policy_version 17110 (0.0008) -[2023-10-15 15:29:10,854][52866] Updated weights for policy 1, policy_version 17160 (0.0007) -[2023-10-15 15:29:11,161][52833] Updated weights for policy 0, policy_version 17120 (0.0007) -[2023-10-15 15:29:11,225][52866] Updated weights for policy 1, policy_version 17170 (0.0009) -[2023-10-15 15:29:11,590][52866] Updated weights for policy 1, policy_version 17180 (0.0009) -[2023-10-15 15:29:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 35127296. Throughput: 0: 1779.5, 1: 1781.9. Samples: 8791986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:29:13,442][51532] Avg episode reward: [(0, '23.550'), (1, '21.540')] -[2023-10-15 15:29:14,865][52833] Updated weights for policy 0, policy_version 17130 (0.0008) -[2023-10-15 15:29:15,237][52833] Updated weights for policy 0, policy_version 17140 (0.0009) -[2023-10-15 15:29:15,311][52866] Updated weights for policy 1, policy_version 17190 (0.0009) -[2023-10-15 15:29:15,599][52833] Updated weights for policy 0, policy_version 17150 (0.0008) -[2023-10-15 15:29:15,682][52866] Updated weights for policy 1, policy_version 17200 (0.0008) -[2023-10-15 15:29:16,052][52866] Updated weights for policy 1, policy_version 17210 (0.0009) -[2023-10-15 15:29:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 35192832. Throughput: 0: 1777.8, 1: 1792.5. Samples: 8802230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:29:18,442][51532] Avg episode reward: [(0, '22.350'), (1, '21.720')] -[2023-10-15 15:29:19,460][52833] Updated weights for policy 0, policy_version 17160 (0.0008) -[2023-10-15 15:29:19,779][52866] Updated weights for policy 1, policy_version 17220 (0.0009) -[2023-10-15 15:29:19,835][52833] Updated weights for policy 0, policy_version 17170 (0.0008) -[2023-10-15 15:29:20,144][52866] Updated weights for policy 1, policy_version 17230 (0.0007) -[2023-10-15 15:29:20,199][52833] Updated weights for policy 0, policy_version 17180 (0.0008) -[2023-10-15 15:29:20,508][52866] Updated weights for policy 1, policy_version 17240 (0.0008) -[2023-10-15 15:29:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 35258368. Throughput: 0: 1778.8, 1: 1791.9. Samples: 8824568. Policy #0 lag: (min: 3.0, avg: 6.0, max: 35.0) -[2023-10-15 15:29:23,442][51532] Avg episode reward: [(0, '21.680'), (1, '22.400')] -[2023-10-15 15:29:23,839][52833] Updated weights for policy 0, policy_version 17190 (0.0008) -[2023-10-15 15:29:24,209][52833] Updated weights for policy 0, policy_version 17200 (0.0009) -[2023-10-15 15:29:24,215][52866] Updated weights for policy 1, policy_version 17250 (0.0007) -[2023-10-15 15:29:24,580][52833] Updated weights for policy 0, policy_version 17210 (0.0008) -[2023-10-15 15:29:24,586][52866] Updated weights for policy 1, policy_version 17260 (0.0009) -[2023-10-15 15:29:24,945][52866] Updated weights for policy 1, policy_version 17270 (0.0008) -[2023-10-15 15:29:25,318][52866] Updated weights for policy 1, policy_version 17280 (0.0009) -[2023-10-15 15:29:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 35323904. Throughput: 0: 1793.8, 1: 1783.9. Samples: 8846764. Policy #0 lag: (min: 3.0, avg: 6.0, max: 35.0) -[2023-10-15 15:29:28,442][51532] Avg episode reward: [(0, '20.650'), (1, '22.050')] -[2023-10-15 15:29:28,451][52833] Updated weights for policy 0, policy_version 17220 (0.0008) -[2023-10-15 15:29:28,816][52833] Updated weights for policy 0, policy_version 17230 (0.0009) -[2023-10-15 15:29:29,157][52866] Updated weights for policy 1, policy_version 17290 (0.0008) -[2023-10-15 15:29:29,179][52833] Updated weights for policy 0, policy_version 17240 (0.0008) -[2023-10-15 15:29:29,528][52866] Updated weights for policy 1, policy_version 17300 (0.0007) -[2023-10-15 15:29:29,895][52866] Updated weights for policy 1, policy_version 17310 (0.0009) -[2023-10-15 15:29:32,910][52833] Updated weights for policy 0, policy_version 17250 (0.0008) -[2023-10-15 15:29:33,280][52833] Updated weights for policy 0, policy_version 17260 (0.0007) -[2023-10-15 15:29:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 35389440. Throughput: 0: 1785.9, 1: 1784.6. Samples: 8856558. Policy #0 lag: (min: 3.0, avg: 6.0, max: 35.0) -[2023-10-15 15:29:33,442][51532] Avg episode reward: [(0, '19.450'), (1, '22.020')] -[2023-10-15 15:29:33,551][52866] Updated weights for policy 1, policy_version 17320 (0.0008) -[2023-10-15 15:29:33,656][52833] Updated weights for policy 0, policy_version 17270 (0.0007) -[2023-10-15 15:29:33,926][52866] Updated weights for policy 1, policy_version 17330 (0.0010) -[2023-10-15 15:29:34,027][52833] Updated weights for policy 0, policy_version 17280 (0.0008) -[2023-10-15 15:29:34,290][52866] Updated weights for policy 1, policy_version 17340 (0.0009) -[2023-10-15 15:29:37,783][52833] Updated weights for policy 0, policy_version 17290 (0.0009) -[2023-10-15 15:29:38,100][52866] Updated weights for policy 1, policy_version 17350 (0.0007) -[2023-10-15 15:29:38,152][52833] Updated weights for policy 0, policy_version 17300 (0.0008) -[2023-10-15 15:29:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 35454976. Throughput: 0: 1789.6, 1: 1786.4. Samples: 8878934. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) -[2023-10-15 15:29:38,441][51532] Avg episode reward: [(0, '21.280'), (1, '23.350')] -[2023-10-15 15:29:38,462][52866] Updated weights for policy 1, policy_version 17360 (0.0007) -[2023-10-15 15:29:38,520][52833] Updated weights for policy 0, policy_version 17310 (0.0010) -[2023-10-15 15:29:38,827][52866] Updated weights for policy 1, policy_version 17370 (0.0010) -[2023-10-15 15:29:42,186][52833] Updated weights for policy 0, policy_version 17320 (0.0008) -[2023-10-15 15:29:42,559][52833] Updated weights for policy 0, policy_version 17330 (0.0008) -[2023-10-15 15:29:42,867][52866] Updated weights for policy 1, policy_version 17380 (0.0009) -[2023-10-15 15:29:42,920][52833] Updated weights for policy 0, policy_version 17340 (0.0008) -[2023-10-15 15:29:43,246][52866] Updated weights for policy 1, policy_version 17390 (0.0008) -[2023-10-15 15:29:43,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 35553280. Throughput: 0: 1799.4, 1: 1809.6. Samples: 8899808. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) -[2023-10-15 15:29:43,441][51532] Avg episode reward: [(0, '23.330'), (1, '24.950')] -[2023-10-15 15:29:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000017344_17760256.pth... -[2023-10-15 15:29:43,485][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000015648_16023552.pth -[2023-10-15 15:29:43,616][52866] Updated weights for policy 1, policy_version 17400 (0.0007) -[2023-10-15 15:29:43,909][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000017408_17825792.pth... -[2023-10-15 15:29:43,938][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000015712_16089088.pth -[2023-10-15 15:29:46,903][52833] Updated weights for policy 0, policy_version 17350 (0.0009) -[2023-10-15 15:29:47,273][52866] Updated weights for policy 1, policy_version 17410 (0.0008) -[2023-10-15 15:29:47,297][52833] Updated weights for policy 0, policy_version 17360 (0.0009) -[2023-10-15 15:29:47,643][52866] Updated weights for policy 1, policy_version 17420 (0.0008) -[2023-10-15 15:29:47,666][52833] Updated weights for policy 0, policy_version 17370 (0.0007) -[2023-10-15 15:29:48,009][52866] Updated weights for policy 1, policy_version 17430 (0.0008) -[2023-10-15 15:29:48,380][52866] Updated weights for policy 1, policy_version 17440 (0.0008) -[2023-10-15 15:29:48,441][51532] Fps is (10 sec: 19660.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 35651584. Throughput: 0: 1783.5, 1: 1783.5. Samples: 8910676. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 15:29:48,442][51532] Avg episode reward: [(0, '24.360'), (1, '25.110')] -[2023-10-15 15:29:51,428][52833] Updated weights for policy 0, policy_version 17380 (0.0009) -[2023-10-15 15:29:51,798][52833] Updated weights for policy 0, policy_version 17390 (0.0007) -[2023-10-15 15:29:52,175][52833] Updated weights for policy 0, policy_version 17400 (0.0008) -[2023-10-15 15:29:52,328][52866] Updated weights for policy 1, policy_version 17450 (0.0007) -[2023-10-15 15:29:52,697][52866] Updated weights for policy 1, policy_version 17460 (0.0010) -[2023-10-15 15:29:53,079][52866] Updated weights for policy 1, policy_version 17470 (0.0010) -[2023-10-15 15:29:53,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35717120. Throughput: 0: 1795.9, 1: 1807.7. Samples: 8931984. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 15:29:53,442][51532] Avg episode reward: [(0, '24.280'), (1, '24.760')] -[2023-10-15 15:29:55,932][52833] Updated weights for policy 0, policy_version 17410 (0.0007) -[2023-10-15 15:29:56,303][52833] Updated weights for policy 0, policy_version 17420 (0.0008) -[2023-10-15 15:29:56,678][52833] Updated weights for policy 0, policy_version 17430 (0.0009) -[2023-10-15 15:29:56,795][52866] Updated weights for policy 1, policy_version 17480 (0.0009) -[2023-10-15 15:29:57,049][52833] Updated weights for policy 0, policy_version 17440 (0.0008) -[2023-10-15 15:29:57,159][52866] Updated weights for policy 1, policy_version 17490 (0.0009) -[2023-10-15 15:29:57,533][52866] Updated weights for policy 1, policy_version 17500 (0.0008) -[2023-10-15 15:29:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 35782656. Throughput: 0: 1782.6, 1: 1781.9. Samples: 8952388. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 15:29:58,442][51532] Avg episode reward: [(0, '25.110'), (1, '26.620')] -[2023-10-15 15:29:58,453][52518] Saving new best policy, reward=26.620! -[2023-10-15 15:30:00,740][52833] Updated weights for policy 0, policy_version 17450 (0.0009) -[2023-10-15 15:30:01,118][52833] Updated weights for policy 0, policy_version 17460 (0.0008) -[2023-10-15 15:30:01,293][52866] Updated weights for policy 1, policy_version 17510 (0.0008) -[2023-10-15 15:30:01,483][52833] Updated weights for policy 0, policy_version 17470 (0.0009) -[2023-10-15 15:30:01,664][52866] Updated weights for policy 1, policy_version 17520 (0.0008) -[2023-10-15 15:30:02,036][52866] Updated weights for policy 1, policy_version 17530 (0.0009) -[2023-10-15 15:30:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 35848192. Throughput: 0: 1805.8, 1: 1805.9. Samples: 8964758. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 15:30:03,442][51532] Avg episode reward: [(0, '24.960'), (1, '26.590')] -[2023-10-15 15:30:05,254][52833] Updated weights for policy 0, policy_version 17480 (0.0007) -[2023-10-15 15:30:05,614][52833] Updated weights for policy 0, policy_version 17490 (0.0007) -[2023-10-15 15:30:05,751][52866] Updated weights for policy 1, policy_version 17540 (0.0007) -[2023-10-15 15:30:05,984][52833] Updated weights for policy 0, policy_version 17500 (0.0007) -[2023-10-15 15:30:06,107][52866] Updated weights for policy 1, policy_version 17550 (0.0008) -[2023-10-15 15:30:06,487][52866] Updated weights for policy 1, policy_version 17560 (0.0009) -[2023-10-15 15:30:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 35913728. Throughput: 0: 1787.1, 1: 1775.9. Samples: 8984904. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 15:30:08,441][51532] Avg episode reward: [(0, '23.780'), (1, '26.700')] -[2023-10-15 15:30:08,442][52518] Saving new best policy, reward=26.700! -[2023-10-15 15:30:09,560][52833] Updated weights for policy 0, policy_version 17510 (0.0007) -[2023-10-15 15:30:09,926][52833] Updated weights for policy 0, policy_version 17520 (0.0007) -[2023-10-15 15:30:10,231][52866] Updated weights for policy 1, policy_version 17570 (0.0007) -[2023-10-15 15:30:10,296][52833] Updated weights for policy 0, policy_version 17530 (0.0010) -[2023-10-15 15:30:10,592][52866] Updated weights for policy 1, policy_version 17580 (0.0007) -[2023-10-15 15:30:10,957][52866] Updated weights for policy 1, policy_version 17590 (0.0008) -[2023-10-15 15:30:11,317][52866] Updated weights for policy 1, policy_version 17600 (0.0008) -[2023-10-15 15:30:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 35979264. Throughput: 0: 1793.6, 1: 1783.3. Samples: 9007724. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 15:30:13,442][51532] Avg episode reward: [(0, '24.790'), (1, '25.000')] -[2023-10-15 15:30:13,964][52833] Updated weights for policy 0, policy_version 17540 (0.0008) -[2023-10-15 15:30:14,349][52833] Updated weights for policy 0, policy_version 17550 (0.0009) -[2023-10-15 15:30:14,716][52833] Updated weights for policy 0, policy_version 17560 (0.0008) -[2023-10-15 15:30:14,984][52866] Updated weights for policy 1, policy_version 17610 (0.0009) -[2023-10-15 15:30:15,345][52866] Updated weights for policy 1, policy_version 17620 (0.0011) -[2023-10-15 15:30:15,712][52866] Updated weights for policy 1, policy_version 17630 (0.0010) -[2023-10-15 15:30:18,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 36044800. Throughput: 0: 1795.2, 1: 1780.9. Samples: 9017482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:30:18,442][51532] Avg episode reward: [(0, '24.310'), (1, '25.920')] -[2023-10-15 15:30:18,606][52833] Updated weights for policy 0, policy_version 17570 (0.0008) -[2023-10-15 15:30:18,969][52833] Updated weights for policy 0, policy_version 17580 (0.0007) -[2023-10-15 15:30:19,335][52833] Updated weights for policy 0, policy_version 17590 (0.0009) -[2023-10-15 15:30:19,562][52866] Updated weights for policy 1, policy_version 17640 (0.0009) -[2023-10-15 15:30:19,697][52833] Updated weights for policy 0, policy_version 17600 (0.0008) -[2023-10-15 15:30:19,931][52866] Updated weights for policy 1, policy_version 17650 (0.0009) -[2023-10-15 15:30:20,295][52866] Updated weights for policy 1, policy_version 17660 (0.0009) -[2023-10-15 15:30:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 36110336. Throughput: 0: 1790.6, 1: 1774.4. Samples: 9039358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:30:23,441][51532] Avg episode reward: [(0, '24.220'), (1, '24.750')] -[2023-10-15 15:30:23,655][52833] Updated weights for policy 0, policy_version 17610 (0.0010) -[2023-10-15 15:30:24,024][52833] Updated weights for policy 0, policy_version 17620 (0.0007) -[2023-10-15 15:30:24,082][52866] Updated weights for policy 1, policy_version 17670 (0.0008) -[2023-10-15 15:30:24,390][52833] Updated weights for policy 0, policy_version 17630 (0.0007) -[2023-10-15 15:30:24,447][52866] Updated weights for policy 1, policy_version 17680 (0.0007) -[2023-10-15 15:30:24,817][52866] Updated weights for policy 1, policy_version 17690 (0.0008) -[2023-10-15 15:30:28,266][52833] Updated weights for policy 0, policy_version 17640 (0.0007) -[2023-10-15 15:30:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 36175872. Throughput: 0: 1805.8, 1: 1781.3. Samples: 9061230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:30:28,442][51532] Avg episode reward: [(0, '22.990'), (1, '24.520')] -[2023-10-15 15:30:28,641][52833] Updated weights for policy 0, policy_version 17650 (0.0009) -[2023-10-15 15:30:28,670][52866] Updated weights for policy 1, policy_version 17700 (0.0008) -[2023-10-15 15:30:29,011][52833] Updated weights for policy 0, policy_version 17660 (0.0008) -[2023-10-15 15:30:29,057][52866] Updated weights for policy 1, policy_version 17710 (0.0009) -[2023-10-15 15:30:29,418][52866] Updated weights for policy 1, policy_version 17720 (0.0008) -[2023-10-15 15:30:32,718][52833] Updated weights for policy 0, policy_version 17670 (0.0007) -[2023-10-15 15:30:33,102][52833] Updated weights for policy 0, policy_version 17680 (0.0008) -[2023-10-15 15:30:33,128][52866] Updated weights for policy 1, policy_version 17730 (0.0007) -[2023-10-15 15:30:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 36241408. Throughput: 0: 1787.6, 1: 1774.5. Samples: 9070972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:30:33,442][51532] Avg episode reward: [(0, '24.810'), (1, '24.000')] -[2023-10-15 15:30:33,476][52833] Updated weights for policy 0, policy_version 17690 (0.0008) -[2023-10-15 15:30:33,486][52866] Updated weights for policy 1, policy_version 17740 (0.0008) -[2023-10-15 15:30:33,862][52866] Updated weights for policy 1, policy_version 17750 (0.0009) -[2023-10-15 15:30:34,225][52866] Updated weights for policy 1, policy_version 17760 (0.0007) -[2023-10-15 15:30:37,262][52833] Updated weights for policy 0, policy_version 17700 (0.0009) -[2023-10-15 15:30:37,639][52833] Updated weights for policy 0, policy_version 17710 (0.0009) -[2023-10-15 15:30:38,004][52833] Updated weights for policy 0, policy_version 17720 (0.0008) -[2023-10-15 15:30:38,160][52866] Updated weights for policy 1, policy_version 17770 (0.0007) -[2023-10-15 15:30:38,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 36339712. Throughput: 0: 1803.5, 1: 1774.6. Samples: 9092996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:30:38,442][51532] Avg episode reward: [(0, '26.350'), (1, '21.820')] -[2023-10-15 15:30:38,526][52866] Updated weights for policy 1, policy_version 17780 (0.0007) -[2023-10-15 15:30:38,888][52866] Updated weights for policy 1, policy_version 17790 (0.0007) -[2023-10-15 15:30:41,778][52833] Updated weights for policy 0, policy_version 17730 (0.0009) -[2023-10-15 15:30:42,147][52833] Updated weights for policy 0, policy_version 17740 (0.0008) -[2023-10-15 15:30:42,504][52833] Updated weights for policy 0, policy_version 17750 (0.0008) -[2023-10-15 15:30:42,752][52866] Updated weights for policy 1, policy_version 17800 (0.0007) -[2023-10-15 15:30:42,876][52833] Updated weights for policy 0, policy_version 17760 (0.0007) -[2023-10-15 15:30:43,119][52866] Updated weights for policy 1, policy_version 17810 (0.0007) -[2023-10-15 15:30:43,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 36405248. Throughput: 0: 1785.6, 1: 1790.7. Samples: 9113322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:30:43,442][51532] Avg episode reward: [(0, '24.680'), (1, '23.290')] -[2023-10-15 15:30:43,490][52866] Updated weights for policy 1, policy_version 17820 (0.0008) -[2023-10-15 15:30:46,585][52833] Updated weights for policy 0, policy_version 17770 (0.0009) -[2023-10-15 15:30:46,952][52833] Updated weights for policy 0, policy_version 17780 (0.0008) -[2023-10-15 15:30:47,298][52866] Updated weights for policy 1, policy_version 17830 (0.0008) -[2023-10-15 15:30:47,320][52833] Updated weights for policy 0, policy_version 17790 (0.0008) -[2023-10-15 15:30:47,669][52866] Updated weights for policy 1, policy_version 17840 (0.0007) -[2023-10-15 15:30:48,030][52866] Updated weights for policy 1, policy_version 17850 (0.0007) -[2023-10-15 15:30:48,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36503552. Throughput: 0: 1790.7, 1: 1772.1. Samples: 9125086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:30:48,442][51532] Avg episode reward: [(0, '24.240'), (1, '23.130')] -[2023-10-15 15:30:51,019][52833] Updated weights for policy 0, policy_version 17800 (0.0007) -[2023-10-15 15:30:51,389][52833] Updated weights for policy 0, policy_version 17810 (0.0009) -[2023-10-15 15:30:51,723][52866] Updated weights for policy 1, policy_version 17860 (0.0008) -[2023-10-15 15:30:51,764][52833] Updated weights for policy 0, policy_version 17820 (0.0009) -[2023-10-15 15:30:52,088][52866] Updated weights for policy 1, policy_version 17870 (0.0007) -[2023-10-15 15:30:52,451][52866] Updated weights for policy 1, policy_version 17880 (0.0008) -[2023-10-15 15:30:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36569088. Throughput: 0: 1776.4, 1: 1791.6. Samples: 9145464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:30:53,442][51532] Avg episode reward: [(0, '26.150'), (1, '24.980')] -[2023-10-15 15:30:55,524][52833] Updated weights for policy 0, policy_version 17830 (0.0007) -[2023-10-15 15:30:55,890][52833] Updated weights for policy 0, policy_version 17840 (0.0007) -[2023-10-15 15:30:56,255][52866] Updated weights for policy 1, policy_version 17890 (0.0007) -[2023-10-15 15:30:56,269][52833] Updated weights for policy 0, policy_version 17850 (0.0009) -[2023-10-15 15:30:56,614][52866] Updated weights for policy 1, policy_version 17900 (0.0008) -[2023-10-15 15:30:56,977][52866] Updated weights for policy 1, policy_version 17910 (0.0008) -[2023-10-15 15:30:57,340][52866] Updated weights for policy 1, policy_version 17920 (0.0008) -[2023-10-15 15:30:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 36634624. Throughput: 0: 1768.0, 1: 1765.4. Samples: 9166728. Policy #0 lag: (min: 1.0, avg: 1.8, max: 16.0) -[2023-10-15 15:30:58,442][51532] Avg episode reward: [(0, '26.640'), (1, '22.930')] -[2023-10-15 15:30:58,451][52410] Saving new best policy, reward=26.640! -[2023-10-15 15:31:00,061][52833] Updated weights for policy 0, policy_version 17860 (0.0011) -[2023-10-15 15:31:00,433][52833] Updated weights for policy 0, policy_version 17870 (0.0009) -[2023-10-15 15:31:00,799][52833] Updated weights for policy 0, policy_version 17880 (0.0007) -[2023-10-15 15:31:01,159][52866] Updated weights for policy 1, policy_version 17930 (0.0009) -[2023-10-15 15:31:01,527][52866] Updated weights for policy 1, policy_version 17940 (0.0007) -[2023-10-15 15:31:01,890][52866] Updated weights for policy 1, policy_version 17950 (0.0009) -[2023-10-15 15:31:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 36700160. Throughput: 0: 1774.8, 1: 1791.6. Samples: 9177974. Policy #0 lag: (min: 1.0, avg: 1.8, max: 16.0) -[2023-10-15 15:31:03,442][51532] Avg episode reward: [(0, '24.610'), (1, '26.590')] -[2023-10-15 15:31:04,535][52833] Updated weights for policy 0, policy_version 17890 (0.0008) -[2023-10-15 15:31:04,897][52833] Updated weights for policy 0, policy_version 17900 (0.0008) -[2023-10-15 15:31:05,282][52833] Updated weights for policy 0, policy_version 17910 (0.0009) -[2023-10-15 15:31:05,464][52866] Updated weights for policy 1, policy_version 17960 (0.0009) -[2023-10-15 15:31:05,649][52833] Updated weights for policy 0, policy_version 17920 (0.0007) -[2023-10-15 15:31:05,833][52866] Updated weights for policy 1, policy_version 17970 (0.0009) -[2023-10-15 15:31:06,207][52866] Updated weights for policy 1, policy_version 17980 (0.0010) -[2023-10-15 15:31:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 36765696. Throughput: 0: 1769.9, 1: 1776.9. Samples: 9198962. Policy #0 lag: (min: 1.0, avg: 1.8, max: 16.0) -[2023-10-15 15:31:08,442][51532] Avg episode reward: [(0, '23.940'), (1, '25.570')] -[2023-10-15 15:31:09,463][52833] Updated weights for policy 0, policy_version 17930 (0.0008) -[2023-10-15 15:31:09,832][52833] Updated weights for policy 0, policy_version 17940 (0.0007) -[2023-10-15 15:31:10,115][52866] Updated weights for policy 1, policy_version 17990 (0.0008) -[2023-10-15 15:31:10,207][52833] Updated weights for policy 0, policy_version 17950 (0.0008) -[2023-10-15 15:31:10,476][52866] Updated weights for policy 1, policy_version 18000 (0.0007) -[2023-10-15 15:31:10,848][52866] Updated weights for policy 1, policy_version 18010 (0.0008) -[2023-10-15 15:31:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 36831232. Throughput: 0: 1778.7, 1: 1774.6. Samples: 9221128. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-15 15:31:13,442][51532] Avg episode reward: [(0, '25.180'), (1, '25.380')] -[2023-10-15 15:31:14,082][52833] Updated weights for policy 0, policy_version 17960 (0.0008) -[2023-10-15 15:31:14,447][52833] Updated weights for policy 0, policy_version 17970 (0.0010) -[2023-10-15 15:31:14,675][52866] Updated weights for policy 1, policy_version 18020 (0.0009) -[2023-10-15 15:31:14,820][52833] Updated weights for policy 0, policy_version 17980 (0.0007) -[2023-10-15 15:31:15,037][52866] Updated weights for policy 1, policy_version 18030 (0.0007) -[2023-10-15 15:31:15,397][52866] Updated weights for policy 1, policy_version 18040 (0.0007) -[2023-10-15 15:31:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 36896768. Throughput: 0: 1773.1, 1: 1781.9. Samples: 9230944. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-15 15:31:18,442][51532] Avg episode reward: [(0, '23.130'), (1, '24.990')] -[2023-10-15 15:31:18,693][52833] Updated weights for policy 0, policy_version 17990 (0.0009) -[2023-10-15 15:31:19,072][52833] Updated weights for policy 0, policy_version 18000 (0.0008) -[2023-10-15 15:31:19,118][52866] Updated weights for policy 1, policy_version 18050 (0.0008) -[2023-10-15 15:31:19,448][52833] Updated weights for policy 0, policy_version 18010 (0.0008) -[2023-10-15 15:31:19,491][52866] Updated weights for policy 1, policy_version 18060 (0.0007) -[2023-10-15 15:31:19,848][52866] Updated weights for policy 1, policy_version 18070 (0.0008) -[2023-10-15 15:31:20,218][52866] Updated weights for policy 1, policy_version 18080 (0.0009) -[2023-10-15 15:31:23,210][52833] Updated weights for policy 0, policy_version 18020 (0.0009) -[2023-10-15 15:31:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 36962304. Throughput: 0: 1770.5, 1: 1786.3. Samples: 9253052. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-15 15:31:23,441][51532] Avg episode reward: [(0, '23.150'), (1, '23.480')] -[2023-10-15 15:31:23,567][52833] Updated weights for policy 0, policy_version 18030 (0.0008) -[2023-10-15 15:31:23,944][52833] Updated weights for policy 0, policy_version 18040 (0.0009) -[2023-10-15 15:31:24,042][52866] Updated weights for policy 1, policy_version 18090 (0.0008) -[2023-10-15 15:31:24,409][52866] Updated weights for policy 1, policy_version 18100 (0.0009) -[2023-10-15 15:31:24,785][52866] Updated weights for policy 1, policy_version 18110 (0.0008) -[2023-10-15 15:31:27,717][52833] Updated weights for policy 0, policy_version 18050 (0.0008) -[2023-10-15 15:31:28,096][52833] Updated weights for policy 0, policy_version 18060 (0.0009) -[2023-10-15 15:31:28,420][52866] Updated weights for policy 1, policy_version 18120 (0.0008) -[2023-10-15 15:31:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 37027840. Throughput: 0: 1798.4, 1: 1804.6. Samples: 9275454. Policy #0 lag: (min: 9.0, avg: 24.0, max: 41.0) -[2023-10-15 15:31:28,442][51532] Avg episode reward: [(0, '22.410'), (1, '25.460')] -[2023-10-15 15:31:28,457][52833] Updated weights for policy 0, policy_version 18070 (0.0007) -[2023-10-15 15:31:28,793][52866] Updated weights for policy 1, policy_version 18130 (0.0009) -[2023-10-15 15:31:28,829][52833] Updated weights for policy 0, policy_version 18080 (0.0007) -[2023-10-15 15:31:29,167][52866] Updated weights for policy 1, policy_version 18140 (0.0009) -[2023-10-15 15:31:32,605][52833] Updated weights for policy 0, policy_version 18090 (0.0008) -[2023-10-15 15:31:32,935][52866] Updated weights for policy 1, policy_version 18150 (0.0009) -[2023-10-15 15:31:32,980][52833] Updated weights for policy 0, policy_version 18100 (0.0010) -[2023-10-15 15:31:33,298][52866] Updated weights for policy 1, policy_version 18160 (0.0007) -[2023-10-15 15:31:33,353][52833] Updated weights for policy 0, policy_version 18110 (0.0008) -[2023-10-15 15:31:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 37126144. Throughput: 0: 1774.8, 1: 1790.8. Samples: 9285534. Policy #0 lag: (min: 9.0, avg: 24.0, max: 41.0) -[2023-10-15 15:31:33,441][51532] Avg episode reward: [(0, '22.940'), (1, '23.500')] -[2023-10-15 15:31:33,669][52866] Updated weights for policy 1, policy_version 18170 (0.0007) -[2023-10-15 15:31:37,171][52833] Updated weights for policy 0, policy_version 18120 (0.0008) -[2023-10-15 15:31:37,334][52866] Updated weights for policy 1, policy_version 18180 (0.0009) -[2023-10-15 15:31:37,533][52833] Updated weights for policy 0, policy_version 18130 (0.0008) -[2023-10-15 15:31:37,687][52866] Updated weights for policy 1, policy_version 18190 (0.0008) -[2023-10-15 15:31:37,904][52833] Updated weights for policy 0, policy_version 18140 (0.0007) -[2023-10-15 15:31:38,058][52866] Updated weights for policy 1, policy_version 18200 (0.0007) -[2023-10-15 15:31:38,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 37224448. Throughput: 0: 1805.5, 1: 1805.8. Samples: 9307972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:31:38,442][51532] Avg episode reward: [(0, '25.000'), (1, '25.220')] -[2023-10-15 15:31:41,588][52833] Updated weights for policy 0, policy_version 18150 (0.0009) -[2023-10-15 15:31:41,727][52866] Updated weights for policy 1, policy_version 18210 (0.0008) -[2023-10-15 15:31:41,958][52833] Updated weights for policy 0, policy_version 18160 (0.0007) -[2023-10-15 15:31:42,094][52866] Updated weights for policy 1, policy_version 18220 (0.0008) -[2023-10-15 15:31:42,316][52833] Updated weights for policy 0, policy_version 18170 (0.0007) -[2023-10-15 15:31:42,449][52866] Updated weights for policy 1, policy_version 18230 (0.0007) -[2023-10-15 15:31:42,819][52866] Updated weights for policy 1, policy_version 18240 (0.0009) -[2023-10-15 15:31:43,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 37289984. Throughput: 0: 1773.1, 1: 1797.8. Samples: 9327422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:31:43,442][51532] Avg episode reward: [(0, '25.500'), (1, '24.640')] -[2023-10-15 15:31:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000018176_18612224.pth... -[2023-10-15 15:31:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000018240_18677760.pth... -[2023-10-15 15:31:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000016544_16941056.pth -[2023-10-15 15:31:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000016512_16908288.pth -[2023-10-15 15:31:46,123][52833] Updated weights for policy 0, policy_version 18180 (0.0009) -[2023-10-15 15:31:46,491][52833] Updated weights for policy 0, policy_version 18190 (0.0007) -[2023-10-15 15:31:46,602][52866] Updated weights for policy 1, policy_version 18250 (0.0007) -[2023-10-15 15:31:46,868][52833] Updated weights for policy 0, policy_version 18200 (0.0007) -[2023-10-15 15:31:46,961][52866] Updated weights for policy 1, policy_version 18260 (0.0007) -[2023-10-15 15:31:47,327][52866] Updated weights for policy 1, policy_version 18270 (0.0009) -[2023-10-15 15:31:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37355520. Throughput: 0: 1797.2, 1: 1806.0. Samples: 9340116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:31:48,442][51532] Avg episode reward: [(0, '25.960'), (1, '26.120')] -[2023-10-15 15:31:50,798][52833] Updated weights for policy 0, policy_version 18210 (0.0007) -[2023-10-15 15:31:50,911][52866] Updated weights for policy 1, policy_version 18280 (0.0007) -[2023-10-15 15:31:51,163][52833] Updated weights for policy 0, policy_version 18220 (0.0008) -[2023-10-15 15:31:51,272][52866] Updated weights for policy 1, policy_version 18290 (0.0010) -[2023-10-15 15:31:51,525][52833] Updated weights for policy 0, policy_version 18230 (0.0007) -[2023-10-15 15:31:51,635][52866] Updated weights for policy 1, policy_version 18300 (0.0008) -[2023-10-15 15:31:51,887][52833] Updated weights for policy 0, policy_version 18240 (0.0007) -[2023-10-15 15:31:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 37421056. Throughput: 0: 1772.9, 1: 1798.5. Samples: 9359674. Policy #0 lag: (min: 7.0, avg: 28.4, max: 32.0) -[2023-10-15 15:31:53,441][51532] Avg episode reward: [(0, '25.760'), (1, '25.420')] -[2023-10-15 15:31:55,521][52833] Updated weights for policy 0, policy_version 18250 (0.0008) -[2023-10-15 15:31:55,602][52866] Updated weights for policy 1, policy_version 18310 (0.0007) -[2023-10-15 15:31:55,887][52833] Updated weights for policy 0, policy_version 18260 (0.0007) -[2023-10-15 15:31:55,978][52866] Updated weights for policy 1, policy_version 18320 (0.0008) -[2023-10-15 15:31:56,264][52833] Updated weights for policy 0, policy_version 18270 (0.0008) -[2023-10-15 15:31:56,347][52866] Updated weights for policy 1, policy_version 18330 (0.0008) -[2023-10-15 15:31:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 37486592. Throughput: 0: 1770.6, 1: 1796.1. Samples: 9381632. Policy #0 lag: (min: 7.0, avg: 28.4, max: 32.0) -[2023-10-15 15:31:58,442][51532] Avg episode reward: [(0, '25.740'), (1, '25.830')] -[2023-10-15 15:32:00,040][52833] Updated weights for policy 0, policy_version 18280 (0.0009) -[2023-10-15 15:32:00,231][52866] Updated weights for policy 1, policy_version 18340 (0.0009) -[2023-10-15 15:32:00,407][52833] Updated weights for policy 0, policy_version 18290 (0.0007) -[2023-10-15 15:32:00,625][52866] Updated weights for policy 1, policy_version 18350 (0.0007) -[2023-10-15 15:32:00,772][52833] Updated weights for policy 0, policy_version 18300 (0.0008) -[2023-10-15 15:32:00,991][52866] Updated weights for policy 1, policy_version 18360 (0.0009) -[2023-10-15 15:32:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 37552128. Throughput: 0: 1773.8, 1: 1798.0. Samples: 9391674. Policy #0 lag: (min: 7.0, avg: 28.4, max: 32.0) -[2023-10-15 15:32:03,442][51532] Avg episode reward: [(0, '25.110'), (1, '26.650')] -[2023-10-15 15:32:04,707][52866] Updated weights for policy 1, policy_version 18370 (0.0009) -[2023-10-15 15:32:04,717][52833] Updated weights for policy 0, policy_version 18310 (0.0007) -[2023-10-15 15:32:05,081][52866] Updated weights for policy 1, policy_version 18380 (0.0008) -[2023-10-15 15:32:05,102][52833] Updated weights for policy 0, policy_version 18320 (0.0008) -[2023-10-15 15:32:05,452][52866] Updated weights for policy 1, policy_version 18390 (0.0009) -[2023-10-15 15:32:05,470][52833] Updated weights for policy 0, policy_version 18330 (0.0008) -[2023-10-15 15:32:05,815][52866] Updated weights for policy 1, policy_version 18400 (0.0008) -[2023-10-15 15:32:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 37617664. Throughput: 0: 1776.4, 1: 1787.7. Samples: 9413436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:32:08,441][51532] Avg episode reward: [(0, '24.110'), (1, '25.230')] -[2023-10-15 15:32:09,121][52833] Updated weights for policy 0, policy_version 18340 (0.0007) -[2023-10-15 15:32:09,483][52833] Updated weights for policy 0, policy_version 18350 (0.0009) -[2023-10-15 15:32:09,499][52866] Updated weights for policy 1, policy_version 18410 (0.0009) -[2023-10-15 15:32:09,846][52833] Updated weights for policy 0, policy_version 18360 (0.0008) -[2023-10-15 15:32:09,858][52866] Updated weights for policy 1, policy_version 18420 (0.0007) -[2023-10-15 15:32:10,227][52866] Updated weights for policy 1, policy_version 18430 (0.0007) -[2023-10-15 15:32:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 37683200. Throughput: 0: 1786.5, 1: 1783.4. Samples: 9436098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:32:13,442][51532] Avg episode reward: [(0, '22.590'), (1, '26.680')] -[2023-10-15 15:32:13,475][52833] Updated weights for policy 0, policy_version 18370 (0.0008) -[2023-10-15 15:32:13,843][52833] Updated weights for policy 0, policy_version 18380 (0.0010) -[2023-10-15 15:32:14,046][52866] Updated weights for policy 1, policy_version 18440 (0.0007) -[2023-10-15 15:32:14,212][52833] Updated weights for policy 0, policy_version 18390 (0.0008) -[2023-10-15 15:32:14,411][52866] Updated weights for policy 1, policy_version 18450 (0.0007) -[2023-10-15 15:32:14,581][52833] Updated weights for policy 0, policy_version 18400 (0.0008) -[2023-10-15 15:32:14,777][52866] Updated weights for policy 1, policy_version 18460 (0.0009) -[2023-10-15 15:32:18,381][52833] Updated weights for policy 0, policy_version 18410 (0.0008) -[2023-10-15 15:32:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 37748736. Throughput: 0: 1781.6, 1: 1780.8. Samples: 9445842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:32:18,441][51532] Avg episode reward: [(0, '21.860'), (1, '26.110')] -[2023-10-15 15:32:18,591][52866] Updated weights for policy 1, policy_version 18470 (0.0008) -[2023-10-15 15:32:18,747][52833] Updated weights for policy 0, policy_version 18420 (0.0007) -[2023-10-15 15:32:18,963][52866] Updated weights for policy 1, policy_version 18480 (0.0007) -[2023-10-15 15:32:19,127][52833] Updated weights for policy 0, policy_version 18430 (0.0007) -[2023-10-15 15:32:19,326][52866] Updated weights for policy 1, policy_version 18490 (0.0008) -[2023-10-15 15:32:22,816][52833] Updated weights for policy 0, policy_version 18440 (0.0008) -[2023-10-15 15:32:23,125][52866] Updated weights for policy 1, policy_version 18500 (0.0008) -[2023-10-15 15:32:23,192][52833] Updated weights for policy 0, policy_version 18450 (0.0007) -[2023-10-15 15:32:23,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 37814272. Throughput: 0: 1785.0, 1: 1772.4. Samples: 9468052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:32:23,441][51532] Avg episode reward: [(0, '22.360'), (1, '26.090')] -[2023-10-15 15:32:23,495][52866] Updated weights for policy 1, policy_version 18510 (0.0010) -[2023-10-15 15:32:23,553][52833] Updated weights for policy 0, policy_version 18460 (0.0007) -[2023-10-15 15:32:23,865][52866] Updated weights for policy 1, policy_version 18520 (0.0008) -[2023-10-15 15:32:27,360][52833] Updated weights for policy 0, policy_version 18470 (0.0008) -[2023-10-15 15:32:27,646][52866] Updated weights for policy 1, policy_version 18530 (0.0009) -[2023-10-15 15:32:27,719][52833] Updated weights for policy 0, policy_version 18480 (0.0008) -[2023-10-15 15:32:28,012][52866] Updated weights for policy 1, policy_version 18540 (0.0009) -[2023-10-15 15:32:28,098][52833] Updated weights for policy 0, policy_version 18490 (0.0007) -[2023-10-15 15:32:28,377][52866] Updated weights for policy 1, policy_version 18550 (0.0007) -[2023-10-15 15:32:28,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 37912576. Throughput: 0: 1800.7, 1: 1791.6. Samples: 9489078. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:32:28,442][51532] Avg episode reward: [(0, '23.350'), (1, '25.610')] -[2023-10-15 15:32:28,751][52866] Updated weights for policy 1, policy_version 18560 (0.0010) -[2023-10-15 15:32:31,823][52833] Updated weights for policy 0, policy_version 18500 (0.0008) -[2023-10-15 15:32:32,186][52833] Updated weights for policy 0, policy_version 18510 (0.0008) -[2023-10-15 15:32:32,551][52833] Updated weights for policy 0, policy_version 18520 (0.0007) -[2023-10-15 15:32:32,585][52866] Updated weights for policy 1, policy_version 18570 (0.0007) -[2023-10-15 15:32:32,950][52866] Updated weights for policy 1, policy_version 18580 (0.0007) -[2023-10-15 15:32:33,319][52866] Updated weights for policy 1, policy_version 18590 (0.0010) -[2023-10-15 15:32:33,441][51532] Fps is (10 sec: 19660.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 38010880. Throughput: 0: 1792.6, 1: 1765.2. Samples: 9500220. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:32:33,442][51532] Avg episode reward: [(0, '22.870'), (1, '25.800')] -[2023-10-15 15:32:36,075][52833] Updated weights for policy 0, policy_version 18530 (0.0008) -[2023-10-15 15:32:36,442][52833] Updated weights for policy 0, policy_version 18540 (0.0008) -[2023-10-15 15:32:36,815][52833] Updated weights for policy 0, policy_version 18550 (0.0008) -[2023-10-15 15:32:37,056][52866] Updated weights for policy 1, policy_version 18600 (0.0010) -[2023-10-15 15:32:37,182][52833] Updated weights for policy 0, policy_version 18560 (0.0008) -[2023-10-15 15:32:37,420][52866] Updated weights for policy 1, policy_version 18610 (0.0007) -[2023-10-15 15:32:37,786][52866] Updated weights for policy 1, policy_version 18620 (0.0009) -[2023-10-15 15:32:38,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38076416. Throughput: 0: 1807.9, 1: 1791.1. Samples: 9521632. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:32:38,442][51532] Avg episode reward: [(0, '23.400'), (1, '25.250')] -[2023-10-15 15:32:41,016][52833] Updated weights for policy 0, policy_version 18570 (0.0009) -[2023-10-15 15:32:41,383][52833] Updated weights for policy 0, policy_version 18580 (0.0009) -[2023-10-15 15:32:41,564][52866] Updated weights for policy 1, policy_version 18630 (0.0007) -[2023-10-15 15:32:41,756][52833] Updated weights for policy 0, policy_version 18590 (0.0007) -[2023-10-15 15:32:41,934][52866] Updated weights for policy 1, policy_version 18640 (0.0009) -[2023-10-15 15:32:42,310][52866] Updated weights for policy 1, policy_version 18650 (0.0008) -[2023-10-15 15:32:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38141952. Throughput: 0: 1802.9, 1: 1772.0. Samples: 9542504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:32:43,442][51532] Avg episode reward: [(0, '23.400'), (1, '23.690')] -[2023-10-15 15:32:45,606][52833] Updated weights for policy 0, policy_version 18600 (0.0007) -[2023-10-15 15:32:45,985][52833] Updated weights for policy 0, policy_version 18610 (0.0007) -[2023-10-15 15:32:46,103][52866] Updated weights for policy 1, policy_version 18660 (0.0008) -[2023-10-15 15:32:46,352][52833] Updated weights for policy 0, policy_version 18620 (0.0009) -[2023-10-15 15:32:46,487][52866] Updated weights for policy 1, policy_version 18670 (0.0009) -[2023-10-15 15:32:46,856][52866] Updated weights for policy 1, policy_version 18680 (0.0009) -[2023-10-15 15:32:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38207488. Throughput: 0: 1816.0, 1: 1798.3. Samples: 9554316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:32:48,442][51532] Avg episode reward: [(0, '21.950'), (1, '24.500')] -[2023-10-15 15:32:50,248][52833] Updated weights for policy 0, policy_version 18630 (0.0008) -[2023-10-15 15:32:50,625][52833] Updated weights for policy 0, policy_version 18640 (0.0007) -[2023-10-15 15:32:50,711][52866] Updated weights for policy 1, policy_version 18690 (0.0009) -[2023-10-15 15:32:51,002][52833] Updated weights for policy 0, policy_version 18650 (0.0008) -[2023-10-15 15:32:51,075][52866] Updated weights for policy 1, policy_version 18700 (0.0008) -[2023-10-15 15:32:51,443][52866] Updated weights for policy 1, policy_version 18710 (0.0008) -[2023-10-15 15:32:51,817][52866] Updated weights for policy 1, policy_version 18720 (0.0007) -[2023-10-15 15:32:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 38273024. Throughput: 0: 1798.6, 1: 1769.9. Samples: 9574020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:32:53,442][51532] Avg episode reward: [(0, '21.420'), (1, '24.980')] -[2023-10-15 15:32:54,604][52833] Updated weights for policy 0, policy_version 18660 (0.0009) -[2023-10-15 15:32:54,965][52833] Updated weights for policy 0, policy_version 18670 (0.0009) -[2023-10-15 15:32:55,347][52833] Updated weights for policy 0, policy_version 18680 (0.0009) -[2023-10-15 15:32:55,626][52866] Updated weights for policy 1, policy_version 18730 (0.0009) -[2023-10-15 15:32:55,994][52866] Updated weights for policy 1, policy_version 18740 (0.0008) -[2023-10-15 15:32:56,368][52866] Updated weights for policy 1, policy_version 18750 (0.0008) -[2023-10-15 15:32:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 38338560. Throughput: 0: 1798.5, 1: 1772.5. Samples: 9596788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:32:58,441][51532] Avg episode reward: [(0, '21.930'), (1, '24.960')] -[2023-10-15 15:32:59,020][52833] Updated weights for policy 0, policy_version 18690 (0.0008) -[2023-10-15 15:32:59,387][52833] Updated weights for policy 0, policy_version 18700 (0.0009) -[2023-10-15 15:32:59,760][52833] Updated weights for policy 0, policy_version 18710 (0.0008) -[2023-10-15 15:33:00,125][52833] Updated weights for policy 0, policy_version 18720 (0.0007) -[2023-10-15 15:33:00,182][52866] Updated weights for policy 1, policy_version 18760 (0.0008) -[2023-10-15 15:33:00,552][52866] Updated weights for policy 1, policy_version 18770 (0.0008) -[2023-10-15 15:33:00,908][52866] Updated weights for policy 1, policy_version 18780 (0.0009) -[2023-10-15 15:33:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 38404096. Throughput: 0: 1802.7, 1: 1778.0. Samples: 9606974. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) -[2023-10-15 15:33:03,441][51532] Avg episode reward: [(0, '21.480'), (1, '25.760')] -[2023-10-15 15:33:03,855][52833] Updated weights for policy 0, policy_version 18730 (0.0009) -[2023-10-15 15:33:04,223][52833] Updated weights for policy 0, policy_version 18740 (0.0010) -[2023-10-15 15:33:04,602][52833] Updated weights for policy 0, policy_version 18750 (0.0008) -[2023-10-15 15:33:04,659][52866] Updated weights for policy 1, policy_version 18790 (0.0007) -[2023-10-15 15:33:05,024][52866] Updated weights for policy 1, policy_version 18800 (0.0007) -[2023-10-15 15:33:05,399][52866] Updated weights for policy 1, policy_version 18810 (0.0008) -[2023-10-15 15:33:08,269][52833] Updated weights for policy 0, policy_version 18760 (0.0008) -[2023-10-15 15:33:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 38469632. Throughput: 0: 1801.6, 1: 1780.9. Samples: 9629268. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) -[2023-10-15 15:33:08,442][51532] Avg episode reward: [(0, '23.730'), (1, '27.120')] -[2023-10-15 15:33:08,442][52518] Saving new best policy, reward=27.120! -[2023-10-15 15:33:08,644][52833] Updated weights for policy 0, policy_version 18770 (0.0008) -[2023-10-15 15:33:09,021][52833] Updated weights for policy 0, policy_version 18780 (0.0007) -[2023-10-15 15:33:09,107][52866] Updated weights for policy 1, policy_version 18820 (0.0010) -[2023-10-15 15:33:09,479][52866] Updated weights for policy 1, policy_version 18830 (0.0010) -[2023-10-15 15:33:09,840][52866] Updated weights for policy 1, policy_version 18840 (0.0007) -[2023-10-15 15:33:12,750][52833] Updated weights for policy 0, policy_version 18790 (0.0010) -[2023-10-15 15:33:13,110][52833] Updated weights for policy 0, policy_version 18800 (0.0011) -[2023-10-15 15:33:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 38535168. Throughput: 0: 1810.5, 1: 1795.3. Samples: 9651336. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) -[2023-10-15 15:33:13,441][51532] Avg episode reward: [(0, '24.430'), (1, '26.180')] -[2023-10-15 15:33:13,488][52833] Updated weights for policy 0, policy_version 18810 (0.0007) -[2023-10-15 15:33:13,689][52866] Updated weights for policy 1, policy_version 18850 (0.0009) -[2023-10-15 15:33:14,061][52866] Updated weights for policy 1, policy_version 18860 (0.0008) -[2023-10-15 15:33:14,432][52866] Updated weights for policy 1, policy_version 18870 (0.0008) -[2023-10-15 15:33:14,792][52866] Updated weights for policy 1, policy_version 18880 (0.0010) -[2023-10-15 15:33:17,313][52833] Updated weights for policy 0, policy_version 18820 (0.0008) -[2023-10-15 15:33:17,672][52833] Updated weights for policy 0, policy_version 18830 (0.0008) -[2023-10-15 15:33:18,039][52833] Updated weights for policy 0, policy_version 18840 (0.0007) -[2023-10-15 15:33:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 38633472. Throughput: 0: 1798.8, 1: 1788.7. Samples: 9661658. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) -[2023-10-15 15:33:18,442][51532] Avg episode reward: [(0, '24.360'), (1, '27.290')] -[2023-10-15 15:33:18,544][52866] Updated weights for policy 1, policy_version 18890 (0.0009) -[2023-10-15 15:33:18,910][52866] Updated weights for policy 1, policy_version 18900 (0.0009) -[2023-10-15 15:33:19,273][52866] Updated weights for policy 1, policy_version 18910 (0.0010) -[2023-10-15 15:33:19,345][52518] Saving new best policy, reward=27.290! -[2023-10-15 15:33:21,818][52833] Updated weights for policy 0, policy_version 18850 (0.0009) -[2023-10-15 15:33:22,185][52833] Updated weights for policy 0, policy_version 18860 (0.0008) -[2023-10-15 15:33:22,565][52833] Updated weights for policy 0, policy_version 18870 (0.0008) -[2023-10-15 15:33:22,938][52833] Updated weights for policy 0, policy_version 18880 (0.0008) -[2023-10-15 15:33:23,017][52866] Updated weights for policy 1, policy_version 18920 (0.0009) -[2023-10-15 15:33:23,383][52866] Updated weights for policy 1, policy_version 18930 (0.0009) -[2023-10-15 15:33:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 38699008. Throughput: 0: 1814.5, 1: 1793.0. Samples: 9683968. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) -[2023-10-15 15:33:23,441][51532] Avg episode reward: [(0, '24.660'), (1, '26.430')] -[2023-10-15 15:33:23,760][52866] Updated weights for policy 1, policy_version 18940 (0.0010) -[2023-10-15 15:33:26,761][52833] Updated weights for policy 0, policy_version 18890 (0.0010) -[2023-10-15 15:33:27,133][52833] Updated weights for policy 0, policy_version 18900 (0.0007) -[2023-10-15 15:33:27,415][52866] Updated weights for policy 1, policy_version 18950 (0.0008) -[2023-10-15 15:33:27,496][52833] Updated weights for policy 0, policy_version 18910 (0.0008) -[2023-10-15 15:33:27,783][52866] Updated weights for policy 1, policy_version 18960 (0.0010) -[2023-10-15 15:33:28,146][52866] Updated weights for policy 1, policy_version 18970 (0.0008) -[2023-10-15 15:33:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 38797312. Throughput: 0: 1791.1, 1: 1800.3. Samples: 9704116. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) -[2023-10-15 15:33:28,442][51532] Avg episode reward: [(0, '24.530'), (1, '27.240')] -[2023-10-15 15:33:31,128][52833] Updated weights for policy 0, policy_version 18920 (0.0008) -[2023-10-15 15:33:31,494][52833] Updated weights for policy 0, policy_version 18930 (0.0008) -[2023-10-15 15:33:31,873][52833] Updated weights for policy 0, policy_version 18940 (0.0007) -[2023-10-15 15:33:32,020][52866] Updated weights for policy 1, policy_version 18980 (0.0010) -[2023-10-15 15:33:32,414][52866] Updated weights for policy 1, policy_version 18990 (0.0008) -[2023-10-15 15:33:32,781][52866] Updated weights for policy 1, policy_version 19000 (0.0009) -[2023-10-15 15:33:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38862848. Throughput: 0: 1813.2, 1: 1788.3. Samples: 9716382. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) -[2023-10-15 15:33:33,441][51532] Avg episode reward: [(0, '23.940'), (1, '25.350')] -[2023-10-15 15:33:35,695][52833] Updated weights for policy 0, policy_version 18950 (0.0008) -[2023-10-15 15:33:36,068][52833] Updated weights for policy 0, policy_version 18960 (0.0009) -[2023-10-15 15:33:36,441][52833] Updated weights for policy 0, policy_version 18970 (0.0009) -[2023-10-15 15:33:36,450][52866] Updated weights for policy 1, policy_version 19010 (0.0010) -[2023-10-15 15:33:36,802][52866] Updated weights for policy 1, policy_version 19020 (0.0007) -[2023-10-15 15:33:37,164][52866] Updated weights for policy 1, policy_version 19030 (0.0008) -[2023-10-15 15:33:37,533][52866] Updated weights for policy 1, policy_version 19040 (0.0011) -[2023-10-15 15:33:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38928384. Throughput: 0: 1801.3, 1: 1810.3. Samples: 9736542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:33:38,442][51532] Avg episode reward: [(0, '22.370'), (1, '26.700')] -[2023-10-15 15:33:40,028][52833] Updated weights for policy 0, policy_version 18980 (0.0007) -[2023-10-15 15:33:40,396][52833] Updated weights for policy 0, policy_version 18990 (0.0008) -[2023-10-15 15:33:40,765][52833] Updated weights for policy 0, policy_version 19000 (0.0008) -[2023-10-15 15:33:41,414][52866] Updated weights for policy 1, policy_version 19050 (0.0007) -[2023-10-15 15:33:41,776][52866] Updated weights for policy 1, policy_version 19060 (0.0009) -[2023-10-15 15:33:42,144][52866] Updated weights for policy 1, policy_version 19070 (0.0011) -[2023-10-15 15:33:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 38993920. Throughput: 0: 1803.2, 1: 1789.1. Samples: 9758442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:33:43,442][51532] Avg episode reward: [(0, '21.380'), (1, '27.110')] -[2023-10-15 15:33:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000019072_19529728.pth... -[2023-10-15 15:33:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000019008_19464192.pth... -[2023-10-15 15:33:43,484][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000017344_17760256.pth -[2023-10-15 15:33:43,492][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000017408_17825792.pth -[2023-10-15 15:33:44,476][52833] Updated weights for policy 0, policy_version 19010 (0.0008) -[2023-10-15 15:33:44,842][52833] Updated weights for policy 0, policy_version 19020 (0.0011) -[2023-10-15 15:33:45,225][52833] Updated weights for policy 0, policy_version 19030 (0.0009) -[2023-10-15 15:33:45,585][52833] Updated weights for policy 0, policy_version 19040 (0.0010) -[2023-10-15 15:33:45,925][52866] Updated weights for policy 1, policy_version 19080 (0.0008) -[2023-10-15 15:33:46,298][52866] Updated weights for policy 1, policy_version 19090 (0.0007) -[2023-10-15 15:33:46,660][52866] Updated weights for policy 1, policy_version 19100 (0.0009) -[2023-10-15 15:33:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 39059456. Throughput: 0: 1797.3, 1: 1809.8. Samples: 9769294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:33:48,441][51532] Avg episode reward: [(0, '22.010'), (1, '27.080')] -[2023-10-15 15:33:49,274][52833] Updated weights for policy 0, policy_version 19050 (0.0009) -[2023-10-15 15:33:49,643][52833] Updated weights for policy 0, policy_version 19060 (0.0011) -[2023-10-15 15:33:50,015][52833] Updated weights for policy 0, policy_version 19070 (0.0011) -[2023-10-15 15:33:50,345][52866] Updated weights for policy 1, policy_version 19110 (0.0009) -[2023-10-15 15:33:50,721][52866] Updated weights for policy 1, policy_version 19120 (0.0010) -[2023-10-15 15:33:51,098][52866] Updated weights for policy 1, policy_version 19130 (0.0011) -[2023-10-15 15:33:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 39124992. Throughput: 0: 1797.1, 1: 1791.8. Samples: 9790772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:33:53,442][51532] Avg episode reward: [(0, '22.870'), (1, '26.890')] -[2023-10-15 15:33:53,698][52833] Updated weights for policy 0, policy_version 19080 (0.0008) -[2023-10-15 15:33:54,071][52833] Updated weights for policy 0, policy_version 19090 (0.0007) -[2023-10-15 15:33:54,450][52833] Updated weights for policy 0, policy_version 19100 (0.0007) -[2023-10-15 15:33:54,754][52866] Updated weights for policy 1, policy_version 19140 (0.0008) -[2023-10-15 15:33:55,127][52866] Updated weights for policy 1, policy_version 19150 (0.0008) -[2023-10-15 15:33:55,488][52866] Updated weights for policy 1, policy_version 19160 (0.0009) -[2023-10-15 15:33:58,170][52833] Updated weights for policy 0, policy_version 19110 (0.0009) -[2023-10-15 15:33:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 39190528. Throughput: 0: 1810.1, 1: 1792.2. Samples: 9813440. Policy #0 lag: (min: 26.0, avg: 31.8, max: 58.0) -[2023-10-15 15:33:58,442][51532] Avg episode reward: [(0, '22.640'), (1, '26.760')] -[2023-10-15 15:33:58,545][52833] Updated weights for policy 0, policy_version 19120 (0.0009) -[2023-10-15 15:33:58,912][52833] Updated weights for policy 0, policy_version 19130 (0.0008) -[2023-10-15 15:33:59,146][52866] Updated weights for policy 1, policy_version 19170 (0.0009) -[2023-10-15 15:33:59,513][52866] Updated weights for policy 1, policy_version 19180 (0.0008) -[2023-10-15 15:33:59,881][52866] Updated weights for policy 1, policy_version 19190 (0.0009) -[2023-10-15 15:34:00,248][52866] Updated weights for policy 1, policy_version 19200 (0.0008) -[2023-10-15 15:34:02,709][52833] Updated weights for policy 0, policy_version 19140 (0.0010) -[2023-10-15 15:34:03,081][52833] Updated weights for policy 0, policy_version 19150 (0.0010) -[2023-10-15 15:34:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 39256064. Throughput: 0: 1801.0, 1: 1792.2. Samples: 9823350. Policy #0 lag: (min: 26.0, avg: 31.8, max: 58.0) -[2023-10-15 15:34:03,442][51532] Avg episode reward: [(0, '24.080'), (1, '28.390')] -[2023-10-15 15:34:03,443][52518] Saving new best policy, reward=28.390! -[2023-10-15 15:34:03,446][52833] Updated weights for policy 0, policy_version 19160 (0.0008) -[2023-10-15 15:34:03,995][52866] Updated weights for policy 1, policy_version 19210 (0.0008) -[2023-10-15 15:34:04,361][52866] Updated weights for policy 1, policy_version 19220 (0.0010) -[2023-10-15 15:34:04,727][52866] Updated weights for policy 1, policy_version 19230 (0.0012) -[2023-10-15 15:34:07,140][52833] Updated weights for policy 0, policy_version 19170 (0.0009) -[2023-10-15 15:34:07,511][52833] Updated weights for policy 0, policy_version 19180 (0.0007) -[2023-10-15 15:34:07,884][52833] Updated weights for policy 0, policy_version 19190 (0.0007) -[2023-10-15 15:34:08,243][52833] Updated weights for policy 0, policy_version 19200 (0.0009) -[2023-10-15 15:34:08,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 39354368. Throughput: 0: 1798.7, 1: 1792.0. Samples: 9845548. Policy #0 lag: (min: 26.0, avg: 31.8, max: 58.0) -[2023-10-15 15:34:08,442][51532] Avg episode reward: [(0, '24.650'), (1, '28.650')] -[2023-10-15 15:34:08,548][52866] Updated weights for policy 1, policy_version 19240 (0.0008) -[2023-10-15 15:34:08,911][52866] Updated weights for policy 1, policy_version 19250 (0.0009) -[2023-10-15 15:34:09,278][52866] Updated weights for policy 1, policy_version 19260 (0.0008) -[2023-10-15 15:34:09,422][52518] Saving new best policy, reward=28.650! -[2023-10-15 15:34:11,928][52833] Updated weights for policy 0, policy_version 19210 (0.0008) -[2023-10-15 15:34:12,299][52833] Updated weights for policy 0, policy_version 19220 (0.0008) -[2023-10-15 15:34:12,676][52833] Updated weights for policy 0, policy_version 19230 (0.0008) -[2023-10-15 15:34:13,041][52866] Updated weights for policy 1, policy_version 19270 (0.0007) -[2023-10-15 15:34:13,399][52866] Updated weights for policy 1, policy_version 19280 (0.0010) -[2023-10-15 15:34:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 39419904. Throughput: 0: 1800.9, 1: 1812.6. Samples: 9866722. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 15:34:13,442][51532] Avg episode reward: [(0, '24.850'), (1, '26.660')] -[2023-10-15 15:34:13,758][52866] Updated weights for policy 1, policy_version 19290 (0.0011) -[2023-10-15 15:34:16,551][52833] Updated weights for policy 0, policy_version 19240 (0.0008) -[2023-10-15 15:34:16,917][52833] Updated weights for policy 0, policy_version 19250 (0.0008) -[2023-10-15 15:34:17,283][52833] Updated weights for policy 0, policy_version 19260 (0.0008) -[2023-10-15 15:34:17,444][52866] Updated weights for policy 1, policy_version 19300 (0.0010) -[2023-10-15 15:34:17,831][52866] Updated weights for policy 1, policy_version 19310 (0.0010) -[2023-10-15 15:34:18,188][52866] Updated weights for policy 1, policy_version 19320 (0.0009) -[2023-10-15 15:34:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 39485440. Throughput: 0: 1798.0, 1: 1796.4. Samples: 9878126. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 15:34:18,441][51532] Avg episode reward: [(0, '26.040'), (1, '25.680')] -[2023-10-15 15:34:21,280][52833] Updated weights for policy 0, policy_version 19270 (0.0009) -[2023-10-15 15:34:21,661][52833] Updated weights for policy 0, policy_version 19280 (0.0010) -[2023-10-15 15:34:22,030][52833] Updated weights for policy 0, policy_version 19290 (0.0010) -[2023-10-15 15:34:22,044][52866] Updated weights for policy 1, policy_version 19330 (0.0010) -[2023-10-15 15:34:22,405][52866] Updated weights for policy 1, policy_version 19340 (0.0008) -[2023-10-15 15:34:22,774][52866] Updated weights for policy 1, policy_version 19350 (0.0008) -[2023-10-15 15:34:23,144][52866] Updated weights for policy 1, policy_version 19360 (0.0007) -[2023-10-15 15:34:23,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 39583744. Throughput: 0: 1804.3, 1: 1807.1. Samples: 9899054. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 15:34:23,442][51532] Avg episode reward: [(0, '24.840'), (1, '26.450')] -[2023-10-15 15:34:25,688][52833] Updated weights for policy 0, policy_version 19300 (0.0008) -[2023-10-15 15:34:26,064][52833] Updated weights for policy 0, policy_version 19310 (0.0008) -[2023-10-15 15:34:26,424][52833] Updated weights for policy 0, policy_version 19320 (0.0011) -[2023-10-15 15:34:26,952][52866] Updated weights for policy 1, policy_version 19370 (0.0008) -[2023-10-15 15:34:27,326][52866] Updated weights for policy 1, policy_version 19380 (0.0007) -[2023-10-15 15:34:27,699][52866] Updated weights for policy 1, policy_version 19390 (0.0008) -[2023-10-15 15:34:28,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 39649280. Throughput: 0: 1788.3, 1: 1797.7. Samples: 9919812. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 15:34:28,442][51532] Avg episode reward: [(0, '24.570'), (1, '27.270')] -[2023-10-15 15:34:30,219][52833] Updated weights for policy 0, policy_version 19330 (0.0008) -[2023-10-15 15:34:30,591][52833] Updated weights for policy 0, policy_version 19340 (0.0008) -[2023-10-15 15:34:30,967][52833] Updated weights for policy 0, policy_version 19350 (0.0007) -[2023-10-15 15:34:31,338][52833] Updated weights for policy 0, policy_version 19360 (0.0009) -[2023-10-15 15:34:31,444][52866] Updated weights for policy 1, policy_version 19400 (0.0008) -[2023-10-15 15:34:31,804][52866] Updated weights for policy 1, policy_version 19410 (0.0007) -[2023-10-15 15:34:32,171][52866] Updated weights for policy 1, policy_version 19420 (0.0007) -[2023-10-15 15:34:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 39714816. Throughput: 0: 1802.2, 1: 1806.0. Samples: 9931666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:34:33,442][51532] Avg episode reward: [(0, '24.590'), (1, '26.310')] -[2023-10-15 15:34:35,106][52833] Updated weights for policy 0, policy_version 19370 (0.0008) -[2023-10-15 15:34:35,485][52833] Updated weights for policy 0, policy_version 19380 (0.0008) -[2023-10-15 15:34:35,857][52833] Updated weights for policy 0, policy_version 19390 (0.0007) -[2023-10-15 15:34:35,914][52866] Updated weights for policy 1, policy_version 19430 (0.0008) -[2023-10-15 15:34:36,280][52866] Updated weights for policy 1, policy_version 19440 (0.0007) -[2023-10-15 15:34:36,649][52866] Updated weights for policy 1, policy_version 19450 (0.0007) -[2023-10-15 15:34:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 39780352. Throughput: 0: 1786.8, 1: 1792.6. Samples: 9951846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:34:38,442][51532] Avg episode reward: [(0, '26.390'), (1, '26.960')] -[2023-10-15 15:34:39,505][52833] Updated weights for policy 0, policy_version 19400 (0.0009) -[2023-10-15 15:34:39,877][52833] Updated weights for policy 0, policy_version 19410 (0.0010) -[2023-10-15 15:34:40,244][52833] Updated weights for policy 0, policy_version 19420 (0.0010) -[2023-10-15 15:34:40,324][52866] Updated weights for policy 1, policy_version 19460 (0.0010) -[2023-10-15 15:34:40,695][52866] Updated weights for policy 1, policy_version 19470 (0.0007) -[2023-10-15 15:34:41,064][52866] Updated weights for policy 1, policy_version 19480 (0.0008) -[2023-10-15 15:34:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 39845888. Throughput: 0: 1788.1, 1: 1797.6. Samples: 9974800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:34:43,442][51532] Avg episode reward: [(0, '28.020'), (1, '25.640')] -[2023-10-15 15:34:43,454][52410] Saving new best policy, reward=28.020! -[2023-10-15 15:34:43,850][52833] Updated weights for policy 0, policy_version 19430 (0.0010) -[2023-10-15 15:34:44,217][52833] Updated weights for policy 0, policy_version 19440 (0.0009) -[2023-10-15 15:34:44,573][52833] Updated weights for policy 0, policy_version 19450 (0.0008) -[2023-10-15 15:34:44,885][52866] Updated weights for policy 1, policy_version 19490 (0.0008) -[2023-10-15 15:34:45,259][52866] Updated weights for policy 1, policy_version 19500 (0.0008) -[2023-10-15 15:34:45,632][52866] Updated weights for policy 1, policy_version 19510 (0.0007) -[2023-10-15 15:34:45,992][52866] Updated weights for policy 1, policy_version 19520 (0.0009) -[2023-10-15 15:34:48,309][52833] Updated weights for policy 0, policy_version 19460 (0.0009) -[2023-10-15 15:34:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 39911424. Throughput: 0: 1787.8, 1: 1796.1. Samples: 9984624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:34:48,442][51532] Avg episode reward: [(0, '27.480'), (1, '29.050')] -[2023-10-15 15:34:48,442][52518] Saving new best policy, reward=29.050! -[2023-10-15 15:34:48,679][52833] Updated weights for policy 0, policy_version 19470 (0.0010) -[2023-10-15 15:34:49,062][52833] Updated weights for policy 0, policy_version 19480 (0.0008) -[2023-10-15 15:34:49,665][52866] Updated weights for policy 1, policy_version 19530 (0.0009) -[2023-10-15 15:34:50,033][52866] Updated weights for policy 1, policy_version 19540 (0.0007) -[2023-10-15 15:34:50,403][52866] Updated weights for policy 1, policy_version 19550 (0.0007) -[2023-10-15 15:34:52,822][52833] Updated weights for policy 0, policy_version 19490 (0.0008) -[2023-10-15 15:34:53,195][52833] Updated weights for policy 0, policy_version 19500 (0.0007) -[2023-10-15 15:34:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 39976960. Throughput: 0: 1792.2, 1: 1792.0. Samples: 10006836. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:34:53,442][51532] Avg episode reward: [(0, '27.560'), (1, '28.390')] -[2023-10-15 15:34:53,572][52833] Updated weights for policy 0, policy_version 19510 (0.0008) -[2023-10-15 15:34:53,945][52833] Updated weights for policy 0, policy_version 19520 (0.0010) -[2023-10-15 15:34:54,217][52866] Updated weights for policy 1, policy_version 19560 (0.0011) -[2023-10-15 15:34:54,589][52866] Updated weights for policy 1, policy_version 19570 (0.0010) -[2023-10-15 15:34:54,961][52866] Updated weights for policy 1, policy_version 19580 (0.0010) -[2023-10-15 15:34:57,620][52833] Updated weights for policy 0, policy_version 19530 (0.0007) -[2023-10-15 15:34:57,991][52833] Updated weights for policy 0, policy_version 19540 (0.0012) -[2023-10-15 15:34:58,368][52833] Updated weights for policy 0, policy_version 19550 (0.0007) -[2023-10-15 15:34:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 40075264. Throughput: 0: 1805.3, 1: 1792.1. Samples: 10028604. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:34:58,441][51532] Avg episode reward: [(0, '26.360'), (1, '27.800')] -[2023-10-15 15:34:58,611][52866] Updated weights for policy 1, policy_version 19590 (0.0010) -[2023-10-15 15:34:58,963][52866] Updated weights for policy 1, policy_version 19600 (0.0010) -[2023-10-15 15:34:59,333][52866] Updated weights for policy 1, policy_version 19610 (0.0011) -[2023-10-15 15:35:02,179][52833] Updated weights for policy 0, policy_version 19560 (0.0008) -[2023-10-15 15:35:02,549][52833] Updated weights for policy 0, policy_version 19570 (0.0008) -[2023-10-15 15:35:02,913][52833] Updated weights for policy 0, policy_version 19580 (0.0007) -[2023-10-15 15:35:03,133][52866] Updated weights for policy 1, policy_version 19620 (0.0009) -[2023-10-15 15:35:03,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 40140800. Throughput: 0: 1789.2, 1: 1788.0. Samples: 10039100. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:35:03,442][51532] Avg episode reward: [(0, '28.300'), (1, '28.190')] -[2023-10-15 15:35:03,443][52410] Saving new best policy, reward=28.300! -[2023-10-15 15:35:03,526][52866] Updated weights for policy 1, policy_version 19630 (0.0007) -[2023-10-15 15:35:03,894][52866] Updated weights for policy 1, policy_version 19640 (0.0011) -[2023-10-15 15:35:06,862][52833] Updated weights for policy 0, policy_version 19590 (0.0007) -[2023-10-15 15:35:07,243][52833] Updated weights for policy 0, policy_version 19600 (0.0007) -[2023-10-15 15:35:07,585][52866] Updated weights for policy 1, policy_version 19650 (0.0009) -[2023-10-15 15:35:07,614][52833] Updated weights for policy 0, policy_version 19610 (0.0009) -[2023-10-15 15:35:07,944][52866] Updated weights for policy 1, policy_version 19660 (0.0008) -[2023-10-15 15:35:08,309][52866] Updated weights for policy 1, policy_version 19670 (0.0007) -[2023-10-15 15:35:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 40206336. Throughput: 0: 1806.2, 1: 1798.6. Samples: 10061270. Policy #0 lag: (min: 34.0, avg: 54.1, max: 56.0) -[2023-10-15 15:35:08,442][51532] Avg episode reward: [(0, '25.360'), (1, '28.520')] -[2023-10-15 15:35:08,670][52866] Updated weights for policy 1, policy_version 19680 (0.0007) -[2023-10-15 15:35:11,360][52833] Updated weights for policy 0, policy_version 19620 (0.0010) -[2023-10-15 15:35:11,725][52833] Updated weights for policy 0, policy_version 19630 (0.0008) -[2023-10-15 15:35:12,103][52833] Updated weights for policy 0, policy_version 19640 (0.0009) -[2023-10-15 15:35:12,405][52866] Updated weights for policy 1, policy_version 19690 (0.0008) -[2023-10-15 15:35:12,762][52866] Updated weights for policy 1, policy_version 19700 (0.0009) -[2023-10-15 15:35:13,135][52866] Updated weights for policy 1, policy_version 19710 (0.0008) -[2023-10-15 15:35:13,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 40304640. Throughput: 0: 1785.8, 1: 1804.8. Samples: 10081390. Policy #0 lag: (min: 34.0, avg: 54.1, max: 56.0) -[2023-10-15 15:35:13,442][51532] Avg episode reward: [(0, '24.620'), (1, '28.670')] -[2023-10-15 15:35:15,964][52833] Updated weights for policy 0, policy_version 19650 (0.0008) -[2023-10-15 15:35:16,335][52833] Updated weights for policy 0, policy_version 19660 (0.0010) -[2023-10-15 15:35:16,695][52833] Updated weights for policy 0, policy_version 19670 (0.0010) -[2023-10-15 15:35:17,058][52866] Updated weights for policy 1, policy_version 19720 (0.0009) -[2023-10-15 15:35:17,061][52833] Updated weights for policy 0, policy_version 19680 (0.0009) -[2023-10-15 15:35:17,423][52866] Updated weights for policy 1, policy_version 19730 (0.0008) -[2023-10-15 15:35:17,787][52866] Updated weights for policy 1, policy_version 19740 (0.0007) -[2023-10-15 15:35:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 40370176. Throughput: 0: 1803.7, 1: 1792.1. Samples: 10093480. Policy #0 lag: (min: 34.0, avg: 54.1, max: 56.0) -[2023-10-15 15:35:18,442][51532] Avg episode reward: [(0, '25.320'), (1, '28.810')] -[2023-10-15 15:35:20,773][52833] Updated weights for policy 0, policy_version 19690 (0.0010) -[2023-10-15 15:35:21,133][52833] Updated weights for policy 0, policy_version 19700 (0.0009) -[2023-10-15 15:35:21,499][52833] Updated weights for policy 0, policy_version 19710 (0.0009) -[2023-10-15 15:35:21,583][52866] Updated weights for policy 1, policy_version 19750 (0.0009) -[2023-10-15 15:35:21,952][52866] Updated weights for policy 1, policy_version 19760 (0.0009) -[2023-10-15 15:35:22,326][52866] Updated weights for policy 1, policy_version 19770 (0.0007) -[2023-10-15 15:35:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40435712. Throughput: 0: 1789.5, 1: 1804.9. Samples: 10113594. Policy #0 lag: (min: 34.0, avg: 54.1, max: 56.0) -[2023-10-15 15:35:23,442][51532] Avg episode reward: [(0, '24.950'), (1, '27.640')] -[2023-10-15 15:35:24,940][52833] Updated weights for policy 0, policy_version 19720 (0.0007) -[2023-10-15 15:35:25,303][52833] Updated weights for policy 0, policy_version 19730 (0.0008) -[2023-10-15 15:35:25,676][52833] Updated weights for policy 0, policy_version 19740 (0.0007) -[2023-10-15 15:35:25,898][52866] Updated weights for policy 1, policy_version 19780 (0.0009) -[2023-10-15 15:35:26,260][52866] Updated weights for policy 1, policy_version 19790 (0.0010) -[2023-10-15 15:35:26,628][52866] Updated weights for policy 1, policy_version 19800 (0.0010) -[2023-10-15 15:35:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40501248. Throughput: 0: 1790.7, 1: 1782.1. Samples: 10135574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:35:28,442][51532] Avg episode reward: [(0, '27.020'), (1, '27.200')] -[2023-10-15 15:35:29,495][52833] Updated weights for policy 0, policy_version 19750 (0.0009) -[2023-10-15 15:35:29,862][52833] Updated weights for policy 0, policy_version 19760 (0.0010) -[2023-10-15 15:35:30,229][52833] Updated weights for policy 0, policy_version 19770 (0.0008) -[2023-10-15 15:35:30,316][52866] Updated weights for policy 1, policy_version 19810 (0.0009) -[2023-10-15 15:35:30,692][52866] Updated weights for policy 1, policy_version 19820 (0.0007) -[2023-10-15 15:35:31,069][52866] Updated weights for policy 1, policy_version 19830 (0.0007) -[2023-10-15 15:35:31,433][52866] Updated weights for policy 1, policy_version 19840 (0.0009) -[2023-10-15 15:35:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 40566784. Throughput: 0: 1788.3, 1: 1798.0. Samples: 10146010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:35:33,442][51532] Avg episode reward: [(0, '24.810'), (1, '26.830')] -[2023-10-15 15:35:33,881][52833] Updated weights for policy 0, policy_version 19780 (0.0008) -[2023-10-15 15:35:34,247][52833] Updated weights for policy 0, policy_version 19790 (0.0008) -[2023-10-15 15:35:34,612][52833] Updated weights for policy 0, policy_version 19800 (0.0010) -[2023-10-15 15:35:35,171][52866] Updated weights for policy 1, policy_version 19850 (0.0009) -[2023-10-15 15:35:35,536][52866] Updated weights for policy 1, policy_version 19860 (0.0010) -[2023-10-15 15:35:35,909][52866] Updated weights for policy 1, policy_version 19870 (0.0009) -[2023-10-15 15:35:38,425][52833] Updated weights for policy 0, policy_version 19810 (0.0008) -[2023-10-15 15:35:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 40632320. Throughput: 0: 1794.4, 1: 1786.8. Samples: 10167990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:35:38,442][51532] Avg episode reward: [(0, '26.300'), (1, '27.860')] -[2023-10-15 15:35:38,793][52833] Updated weights for policy 0, policy_version 19820 (0.0009) -[2023-10-15 15:35:39,163][52833] Updated weights for policy 0, policy_version 19830 (0.0010) -[2023-10-15 15:35:39,535][52833] Updated weights for policy 0, policy_version 19840 (0.0010) -[2023-10-15 15:35:39,788][52866] Updated weights for policy 1, policy_version 19880 (0.0007) -[2023-10-15 15:35:40,160][52866] Updated weights for policy 1, policy_version 19890 (0.0008) -[2023-10-15 15:35:40,527][52866] Updated weights for policy 1, policy_version 19900 (0.0008) -[2023-10-15 15:35:43,409][52833] Updated weights for policy 0, policy_version 19850 (0.0007) -[2023-10-15 15:35:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 40697856. Throughput: 0: 1807.7, 1: 1786.3. Samples: 10190336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:35:43,442][51532] Avg episode reward: [(0, '24.090'), (1, '27.860')] -[2023-10-15 15:35:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000019904_20381696.pth... -[2023-10-15 15:35:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000018240_18677760.pth -[2023-10-15 15:35:43,770][52833] Updated weights for policy 0, policy_version 19860 (0.0008) -[2023-10-15 15:35:44,138][52833] Updated weights for policy 0, policy_version 19870 (0.0007) -[2023-10-15 15:35:44,212][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000019872_20348928.pth... -[2023-10-15 15:35:44,240][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000018176_18612224.pth -[2023-10-15 15:35:44,298][52866] Updated weights for policy 1, policy_version 19910 (0.0009) -[2023-10-15 15:35:44,669][52866] Updated weights for policy 1, policy_version 19920 (0.0008) -[2023-10-15 15:35:45,039][52866] Updated weights for policy 1, policy_version 19930 (0.0007) -[2023-10-15 15:35:47,819][52833] Updated weights for policy 0, policy_version 19880 (0.0008) -[2023-10-15 15:35:48,202][52833] Updated weights for policy 0, policy_version 19890 (0.0009) -[2023-10-15 15:35:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 40763392. Throughput: 0: 1794.2, 1: 1786.4. Samples: 10200228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:35:48,441][51532] Avg episode reward: [(0, '23.260'), (1, '28.070')] -[2023-10-15 15:35:48,576][52833] Updated weights for policy 0, policy_version 19900 (0.0010) -[2023-10-15 15:35:48,883][52866] Updated weights for policy 1, policy_version 19940 (0.0008) -[2023-10-15 15:35:49,262][52866] Updated weights for policy 1, policy_version 19950 (0.0008) -[2023-10-15 15:35:49,628][52866] Updated weights for policy 1, policy_version 19960 (0.0009) -[2023-10-15 15:35:52,379][52833] Updated weights for policy 0, policy_version 19910 (0.0009) -[2023-10-15 15:35:52,766][52833] Updated weights for policy 0, policy_version 19920 (0.0009) -[2023-10-15 15:35:53,129][52833] Updated weights for policy 0, policy_version 19930 (0.0010) -[2023-10-15 15:35:53,376][52866] Updated weights for policy 1, policy_version 19970 (0.0011) -[2023-10-15 15:35:53,441][51532] Fps is (10 sec: 16384.7, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 40861696. Throughput: 0: 1807.7, 1: 1774.6. Samples: 10222474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:35:53,441][51532] Avg episode reward: [(0, '26.010'), (1, '28.400')] -[2023-10-15 15:35:53,743][52866] Updated weights for policy 1, policy_version 19980 (0.0010) -[2023-10-15 15:35:54,103][52866] Updated weights for policy 1, policy_version 19990 (0.0008) -[2023-10-15 15:35:54,472][52866] Updated weights for policy 1, policy_version 20000 (0.0010) -[2023-10-15 15:35:56,843][52833] Updated weights for policy 0, policy_version 19940 (0.0010) -[2023-10-15 15:35:57,223][52833] Updated weights for policy 0, policy_version 19950 (0.0010) -[2023-10-15 15:35:57,592][52833] Updated weights for policy 0, policy_version 19960 (0.0009) -[2023-10-15 15:35:58,261][52866] Updated weights for policy 1, policy_version 20010 (0.0010) -[2023-10-15 15:35:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 40927232. Throughput: 0: 1799.0, 1: 1800.9. Samples: 10243388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:35:58,441][51532] Avg episode reward: [(0, '26.450'), (1, '28.750')] -[2023-10-15 15:35:58,634][52866] Updated weights for policy 1, policy_version 20020 (0.0009) -[2023-10-15 15:35:59,001][52866] Updated weights for policy 1, policy_version 20030 (0.0010) -[2023-10-15 15:36:01,223][52833] Updated weights for policy 0, policy_version 19970 (0.0009) -[2023-10-15 15:36:01,593][52833] Updated weights for policy 0, policy_version 19980 (0.0011) -[2023-10-15 15:36:01,970][52833] Updated weights for policy 0, policy_version 19990 (0.0009) -[2023-10-15 15:36:02,345][52833] Updated weights for policy 0, policy_version 20000 (0.0008) -[2023-10-15 15:36:02,693][52866] Updated weights for policy 1, policy_version 20040 (0.0010) -[2023-10-15 15:36:03,056][52866] Updated weights for policy 1, policy_version 20050 (0.0010) -[2023-10-15 15:36:03,426][52866] Updated weights for policy 1, policy_version 20060 (0.0011) -[2023-10-15 15:36:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 40992768. Throughput: 0: 1803.4, 1: 1782.5. Samples: 10254844. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:36:03,441][51532] Avg episode reward: [(0, '26.900'), (1, '28.710')] -[2023-10-15 15:36:06,075][52833] Updated weights for policy 0, policy_version 20010 (0.0009) -[2023-10-15 15:36:06,445][52833] Updated weights for policy 0, policy_version 20020 (0.0007) -[2023-10-15 15:36:06,812][52833] Updated weights for policy 0, policy_version 20030 (0.0008) -[2023-10-15 15:36:07,134][52866] Updated weights for policy 1, policy_version 20070 (0.0011) -[2023-10-15 15:36:07,494][52866] Updated weights for policy 1, policy_version 20080 (0.0010) -[2023-10-15 15:36:07,861][52866] Updated weights for policy 1, policy_version 20090 (0.0010) -[2023-10-15 15:36:08,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 41091072. Throughput: 0: 1806.1, 1: 1804.3. Samples: 10276064. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:36:08,442][51532] Avg episode reward: [(0, '26.990'), (1, '28.940')] -[2023-10-15 15:36:10,502][52833] Updated weights for policy 0, policy_version 20040 (0.0009) -[2023-10-15 15:36:10,872][52833] Updated weights for policy 0, policy_version 20050 (0.0011) -[2023-10-15 15:36:11,234][52833] Updated weights for policy 0, policy_version 20060 (0.0009) -[2023-10-15 15:36:11,703][52866] Updated weights for policy 1, policy_version 20100 (0.0010) -[2023-10-15 15:36:12,060][52866] Updated weights for policy 1, policy_version 20110 (0.0011) -[2023-10-15 15:36:12,429][52866] Updated weights for policy 1, policy_version 20120 (0.0009) -[2023-10-15 15:36:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41156608. Throughput: 0: 1802.2, 1: 1788.6. Samples: 10297160. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:36:13,441][51532] Avg episode reward: [(0, '27.470'), (1, '29.000')] -[2023-10-15 15:36:14,917][52833] Updated weights for policy 0, policy_version 20070 (0.0009) -[2023-10-15 15:36:15,286][52833] Updated weights for policy 0, policy_version 20080 (0.0008) -[2023-10-15 15:36:15,658][52833] Updated weights for policy 0, policy_version 20090 (0.0008) -[2023-10-15 15:36:16,158][52866] Updated weights for policy 1, policy_version 20130 (0.0007) -[2023-10-15 15:36:16,512][52866] Updated weights for policy 1, policy_version 20140 (0.0008) -[2023-10-15 15:36:16,876][52866] Updated weights for policy 1, policy_version 20150 (0.0007) -[2023-10-15 15:36:17,247][52866] Updated weights for policy 1, policy_version 20160 (0.0007) -[2023-10-15 15:36:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41222144. Throughput: 0: 1804.9, 1: 1808.0. Samples: 10308590. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 15:36:18,441][51532] Avg episode reward: [(0, '28.940'), (1, '27.880')] -[2023-10-15 15:36:18,442][52410] Saving new best policy, reward=28.940! -[2023-10-15 15:36:19,480][52833] Updated weights for policy 0, policy_version 20100 (0.0008) -[2023-10-15 15:36:19,846][52833] Updated weights for policy 0, policy_version 20110 (0.0011) -[2023-10-15 15:36:20,212][52833] Updated weights for policy 0, policy_version 20120 (0.0010) -[2023-10-15 15:36:20,978][52866] Updated weights for policy 1, policy_version 20170 (0.0008) -[2023-10-15 15:36:21,343][52866] Updated weights for policy 1, policy_version 20180 (0.0009) -[2023-10-15 15:36:21,720][52866] Updated weights for policy 1, policy_version 20190 (0.0009) -[2023-10-15 15:36:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 41287680. Throughput: 0: 1795.5, 1: 1793.3. Samples: 10329486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:36:23,442][51532] Avg episode reward: [(0, '27.010'), (1, '27.440')] -[2023-10-15 15:36:24,109][52833] Updated weights for policy 0, policy_version 20130 (0.0009) -[2023-10-15 15:36:24,480][52833] Updated weights for policy 0, policy_version 20140 (0.0008) -[2023-10-15 15:36:24,847][52833] Updated weights for policy 0, policy_version 20150 (0.0008) -[2023-10-15 15:36:25,216][52833] Updated weights for policy 0, policy_version 20160 (0.0008) -[2023-10-15 15:36:25,564][52866] Updated weights for policy 1, policy_version 20200 (0.0008) -[2023-10-15 15:36:25,939][52866] Updated weights for policy 1, policy_version 20210 (0.0007) -[2023-10-15 15:36:26,295][52866] Updated weights for policy 1, policy_version 20220 (0.0007) -[2023-10-15 15:36:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 41353216. Throughput: 0: 1794.2, 1: 1787.6. Samples: 10351514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:36:28,442][51532] Avg episode reward: [(0, '27.510'), (1, '26.110')] -[2023-10-15 15:36:28,904][52833] Updated weights for policy 0, policy_version 20170 (0.0008) -[2023-10-15 15:36:29,271][52833] Updated weights for policy 0, policy_version 20180 (0.0008) -[2023-10-15 15:36:29,637][52833] Updated weights for policy 0, policy_version 20190 (0.0008) -[2023-10-15 15:36:30,093][52866] Updated weights for policy 1, policy_version 20230 (0.0011) -[2023-10-15 15:36:30,462][52866] Updated weights for policy 1, policy_version 20240 (0.0009) -[2023-10-15 15:36:30,827][52866] Updated weights for policy 1, policy_version 20250 (0.0008) -[2023-10-15 15:36:33,335][52833] Updated weights for policy 0, policy_version 20200 (0.0009) -[2023-10-15 15:36:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 41418752. Throughput: 0: 1791.2, 1: 1790.7. Samples: 10361416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:36:33,441][51532] Avg episode reward: [(0, '26.450'), (1, '26.640')] -[2023-10-15 15:36:33,714][52833] Updated weights for policy 0, policy_version 20210 (0.0008) -[2023-10-15 15:36:34,079][52833] Updated weights for policy 0, policy_version 20220 (0.0009) -[2023-10-15 15:36:34,501][52866] Updated weights for policy 1, policy_version 20260 (0.0008) -[2023-10-15 15:36:34,895][52866] Updated weights for policy 1, policy_version 20270 (0.0008) -[2023-10-15 15:36:35,271][52866] Updated weights for policy 1, policy_version 20280 (0.0007) -[2023-10-15 15:36:37,730][52833] Updated weights for policy 0, policy_version 20230 (0.0007) -[2023-10-15 15:36:38,094][52833] Updated weights for policy 0, policy_version 20240 (0.0007) -[2023-10-15 15:36:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 41484288. Throughput: 0: 1787.9, 1: 1791.7. Samples: 10383558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:36:38,442][51532] Avg episode reward: [(0, '27.610'), (1, '28.340')] -[2023-10-15 15:36:38,465][52833] Updated weights for policy 0, policy_version 20250 (0.0007) -[2023-10-15 15:36:39,056][52866] Updated weights for policy 1, policy_version 20290 (0.0009) -[2023-10-15 15:36:39,430][52866] Updated weights for policy 1, policy_version 20300 (0.0011) -[2023-10-15 15:36:39,800][52866] Updated weights for policy 1, policy_version 20310 (0.0008) -[2023-10-15 15:36:40,162][52866] Updated weights for policy 1, policy_version 20320 (0.0011) -[2023-10-15 15:36:42,187][52833] Updated weights for policy 0, policy_version 20260 (0.0009) -[2023-10-15 15:36:42,549][52833] Updated weights for policy 0, policy_version 20270 (0.0009) -[2023-10-15 15:36:42,923][52833] Updated weights for policy 0, policy_version 20280 (0.0007) -[2023-10-15 15:36:43,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 41582592. Throughput: 0: 1806.8, 1: 1791.1. Samples: 10405290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:36:43,442][51532] Avg episode reward: [(0, '27.450'), (1, '29.140')] -[2023-10-15 15:36:43,454][52518] Saving new best policy, reward=29.140! -[2023-10-15 15:36:43,991][52866] Updated weights for policy 1, policy_version 20330 (0.0007) -[2023-10-15 15:36:44,362][52866] Updated weights for policy 1, policy_version 20340 (0.0008) -[2023-10-15 15:36:44,726][52866] Updated weights for policy 1, policy_version 20350 (0.0011) -[2023-10-15 15:36:46,758][52833] Updated weights for policy 0, policy_version 20290 (0.0009) -[2023-10-15 15:36:47,128][52833] Updated weights for policy 0, policy_version 20300 (0.0007) -[2023-10-15 15:36:47,492][52833] Updated weights for policy 0, policy_version 20310 (0.0009) -[2023-10-15 15:36:47,863][52833] Updated weights for policy 0, policy_version 20320 (0.0011) -[2023-10-15 15:36:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 41648128. Throughput: 0: 1793.8, 1: 1785.0. Samples: 10415890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:36:48,441][51532] Avg episode reward: [(0, '28.430'), (1, '27.240')] -[2023-10-15 15:36:48,583][52866] Updated weights for policy 1, policy_version 20360 (0.0008) -[2023-10-15 15:36:48,935][52866] Updated weights for policy 1, policy_version 20370 (0.0007) -[2023-10-15 15:36:49,303][52866] Updated weights for policy 1, policy_version 20380 (0.0007) -[2023-10-15 15:36:51,607][52833] Updated weights for policy 0, policy_version 20330 (0.0010) -[2023-10-15 15:36:51,974][52833] Updated weights for policy 0, policy_version 20340 (0.0007) -[2023-10-15 15:36:52,340][52833] Updated weights for policy 0, policy_version 20350 (0.0007) -[2023-10-15 15:36:53,044][52866] Updated weights for policy 1, policy_version 20390 (0.0010) -[2023-10-15 15:36:53,415][52866] Updated weights for policy 1, policy_version 20400 (0.0007) -[2023-10-15 15:36:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 41713664. Throughput: 0: 1803.7, 1: 1787.8. Samples: 10437682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:36:53,442][51532] Avg episode reward: [(0, '27.680'), (1, '27.930')] -[2023-10-15 15:36:53,773][52866] Updated weights for policy 1, policy_version 20410 (0.0008) -[2023-10-15 15:36:56,068][52833] Updated weights for policy 0, policy_version 20360 (0.0008) -[2023-10-15 15:36:56,441][52833] Updated weights for policy 0, policy_version 20370 (0.0008) -[2023-10-15 15:36:56,808][52833] Updated weights for policy 0, policy_version 20380 (0.0007) -[2023-10-15 15:36:57,491][52866] Updated weights for policy 1, policy_version 20420 (0.0010) -[2023-10-15 15:36:57,862][52866] Updated weights for policy 1, policy_version 20430 (0.0008) -[2023-10-15 15:36:58,230][52866] Updated weights for policy 1, policy_version 20440 (0.0009) -[2023-10-15 15:36:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 41779200. Throughput: 0: 1786.7, 1: 1803.9. Samples: 10458738. Policy #0 lag: (min: 18.0, avg: 24.6, max: 50.0) -[2023-10-15 15:36:58,442][51532] Avg episode reward: [(0, '27.260'), (1, '28.520')] -[2023-10-15 15:37:00,545][52833] Updated weights for policy 0, policy_version 20390 (0.0009) -[2023-10-15 15:37:00,917][52833] Updated weights for policy 0, policy_version 20400 (0.0011) -[2023-10-15 15:37:01,287][52833] Updated weights for policy 0, policy_version 20410 (0.0008) -[2023-10-15 15:37:01,967][52866] Updated weights for policy 1, policy_version 20450 (0.0009) -[2023-10-15 15:37:02,324][52866] Updated weights for policy 1, policy_version 20460 (0.0008) -[2023-10-15 15:37:02,686][52866] Updated weights for policy 1, policy_version 20470 (0.0009) -[2023-10-15 15:37:03,054][52866] Updated weights for policy 1, policy_version 20480 (0.0007) -[2023-10-15 15:37:03,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 41877504. Throughput: 0: 1803.4, 1: 1788.0. Samples: 10470206. Policy #0 lag: (min: 18.0, avg: 24.6, max: 50.0) -[2023-10-15 15:37:03,441][51532] Avg episode reward: [(0, '25.140'), (1, '31.930')] -[2023-10-15 15:37:03,442][52518] Saving new best policy, reward=31.930! -[2023-10-15 15:37:05,012][52833] Updated weights for policy 0, policy_version 20420 (0.0009) -[2023-10-15 15:37:05,384][52833] Updated weights for policy 0, policy_version 20430 (0.0007) -[2023-10-15 15:37:05,749][52833] Updated weights for policy 0, policy_version 20440 (0.0007) -[2023-10-15 15:37:06,790][52866] Updated weights for policy 1, policy_version 20490 (0.0007) -[2023-10-15 15:37:07,157][52866] Updated weights for policy 1, policy_version 20500 (0.0008) -[2023-10-15 15:37:07,519][52866] Updated weights for policy 1, policy_version 20510 (0.0010) -[2023-10-15 15:37:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41943040. Throughput: 0: 1789.9, 1: 1801.7. Samples: 10491110. Policy #0 lag: (min: 18.0, avg: 24.6, max: 50.0) -[2023-10-15 15:37:08,442][51532] Avg episode reward: [(0, '24.910'), (1, '29.850')] -[2023-10-15 15:37:09,498][52833] Updated weights for policy 0, policy_version 20450 (0.0009) -[2023-10-15 15:37:09,871][52833] Updated weights for policy 0, policy_version 20460 (0.0008) -[2023-10-15 15:37:10,233][52833] Updated weights for policy 0, policy_version 20470 (0.0009) -[2023-10-15 15:37:10,598][52833] Updated weights for policy 0, policy_version 20480 (0.0010) -[2023-10-15 15:37:11,243][52866] Updated weights for policy 1, policy_version 20520 (0.0008) -[2023-10-15 15:37:11,612][52866] Updated weights for policy 1, policy_version 20530 (0.0009) -[2023-10-15 15:37:11,977][52866] Updated weights for policy 1, policy_version 20540 (0.0010) -[2023-10-15 15:37:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 42008576. Throughput: 0: 1797.6, 1: 1790.5. Samples: 10512978. Policy #0 lag: (min: 18.0, avg: 24.6, max: 50.0) -[2023-10-15 15:37:13,442][51532] Avg episode reward: [(0, '23.780'), (1, '28.090')] -[2023-10-15 15:37:14,283][52833] Updated weights for policy 0, policy_version 20490 (0.0010) -[2023-10-15 15:37:14,657][52833] Updated weights for policy 0, policy_version 20500 (0.0010) -[2023-10-15 15:37:15,020][52833] Updated weights for policy 0, policy_version 20510 (0.0010) -[2023-10-15 15:37:15,720][52866] Updated weights for policy 1, policy_version 20550 (0.0009) -[2023-10-15 15:37:16,090][52866] Updated weights for policy 1, policy_version 20560 (0.0010) -[2023-10-15 15:37:16,455][52866] Updated weights for policy 1, policy_version 20570 (0.0010) -[2023-10-15 15:37:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 42074112. Throughput: 0: 1796.6, 1: 1809.2. Samples: 10523674. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 15:37:18,442][51532] Avg episode reward: [(0, '23.730'), (1, '27.460')] -[2023-10-15 15:37:18,798][52833] Updated weights for policy 0, policy_version 20520 (0.0008) -[2023-10-15 15:37:19,170][52833] Updated weights for policy 0, policy_version 20530 (0.0007) -[2023-10-15 15:37:19,544][52833] Updated weights for policy 0, policy_version 20540 (0.0008) -[2023-10-15 15:37:20,305][52866] Updated weights for policy 1, policy_version 20580 (0.0010) -[2023-10-15 15:37:20,676][52866] Updated weights for policy 1, policy_version 20590 (0.0010) -[2023-10-15 15:37:21,050][52866] Updated weights for policy 1, policy_version 20600 (0.0010) -[2023-10-15 15:37:23,383][52833] Updated weights for policy 0, policy_version 20550 (0.0009) -[2023-10-15 15:37:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 42139648. Throughput: 0: 1799.4, 1: 1793.6. Samples: 10545244. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 15:37:23,441][51532] Avg episode reward: [(0, '22.400'), (1, '27.740')] -[2023-10-15 15:37:23,769][52833] Updated weights for policy 0, policy_version 20560 (0.0009) -[2023-10-15 15:37:24,130][52833] Updated weights for policy 0, policy_version 20570 (0.0009) -[2023-10-15 15:37:24,666][52866] Updated weights for policy 1, policy_version 20610 (0.0009) -[2023-10-15 15:37:25,064][52866] Updated weights for policy 1, policy_version 20620 (0.0008) -[2023-10-15 15:37:25,433][52866] Updated weights for policy 1, policy_version 20630 (0.0008) -[2023-10-15 15:37:25,808][52866] Updated weights for policy 1, policy_version 20640 (0.0007) -[2023-10-15 15:37:27,922][52833] Updated weights for policy 0, policy_version 20580 (0.0008) -[2023-10-15 15:37:28,286][52833] Updated weights for policy 0, policy_version 20590 (0.0008) -[2023-10-15 15:37:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 42205184. Throughput: 0: 1808.3, 1: 1796.3. Samples: 10567494. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 15:37:28,442][51532] Avg episode reward: [(0, '23.470'), (1, '25.470')] -[2023-10-15 15:37:28,655][52833] Updated weights for policy 0, policy_version 20600 (0.0007) -[2023-10-15 15:37:29,477][52866] Updated weights for policy 1, policy_version 20650 (0.0011) -[2023-10-15 15:37:29,835][52866] Updated weights for policy 1, policy_version 20660 (0.0009) -[2023-10-15 15:37:30,204][52866] Updated weights for policy 1, policy_version 20670 (0.0007) -[2023-10-15 15:37:32,382][52833] Updated weights for policy 0, policy_version 20610 (0.0008) -[2023-10-15 15:37:32,756][52833] Updated weights for policy 0, policy_version 20620 (0.0010) -[2023-10-15 15:37:33,125][52833] Updated weights for policy 0, policy_version 20630 (0.0008) -[2023-10-15 15:37:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 42270720. Throughput: 0: 1789.5, 1: 1800.2. Samples: 10577428. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 15:37:33,442][51532] Avg episode reward: [(0, '26.580'), (1, '23.550')] -[2023-10-15 15:37:33,496][52833] Updated weights for policy 0, policy_version 20640 (0.0007) -[2023-10-15 15:37:33,932][52866] Updated weights for policy 1, policy_version 20680 (0.0008) -[2023-10-15 15:37:34,300][52866] Updated weights for policy 1, policy_version 20690 (0.0007) -[2023-10-15 15:37:34,669][52866] Updated weights for policy 1, policy_version 20700 (0.0009) -[2023-10-15 15:37:37,348][52833] Updated weights for policy 0, policy_version 20650 (0.0011) -[2023-10-15 15:37:37,711][52833] Updated weights for policy 0, policy_version 20660 (0.0011) -[2023-10-15 15:37:38,077][52833] Updated weights for policy 0, policy_version 20670 (0.0008) -[2023-10-15 15:37:38,392][52866] Updated weights for policy 1, policy_version 20710 (0.0011) -[2023-10-15 15:37:38,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 42369024. Throughput: 0: 1802.7, 1: 1799.2. Samples: 10599766. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-15 15:37:38,441][51532] Avg episode reward: [(0, '26.510'), (1, '25.030')] -[2023-10-15 15:37:38,758][52866] Updated weights for policy 1, policy_version 20720 (0.0011) -[2023-10-15 15:37:39,130][52866] Updated weights for policy 1, policy_version 20730 (0.0009) -[2023-10-15 15:37:41,777][52833] Updated weights for policy 0, policy_version 20680 (0.0008) -[2023-10-15 15:37:42,159][52833] Updated weights for policy 0, policy_version 20690 (0.0009) -[2023-10-15 15:37:42,520][52833] Updated weights for policy 0, policy_version 20700 (0.0008) -[2023-10-15 15:37:42,842][52866] Updated weights for policy 1, policy_version 20740 (0.0009) -[2023-10-15 15:37:43,202][52866] Updated weights for policy 1, policy_version 20750 (0.0007) -[2023-10-15 15:37:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 42434560. Throughput: 0: 1786.9, 1: 1809.2. Samples: 10620562. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-15 15:37:43,442][51532] Avg episode reward: [(0, '25.850'), (1, '25.970')] -[2023-10-15 15:37:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000020704_21200896.pth... -[2023-10-15 15:37:43,483][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000019008_19464192.pth -[2023-10-15 15:37:43,561][52866] Updated weights for policy 1, policy_version 20760 (0.0008) -[2023-10-15 15:37:43,845][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000020768_21266432.pth... -[2023-10-15 15:37:43,874][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000019072_19529728.pth -[2023-10-15 15:37:46,222][52833] Updated weights for policy 0, policy_version 20710 (0.0007) -[2023-10-15 15:37:46,588][52833] Updated weights for policy 0, policy_version 20720 (0.0008) -[2023-10-15 15:37:46,954][52833] Updated weights for policy 0, policy_version 20730 (0.0009) -[2023-10-15 15:37:47,348][52866] Updated weights for policy 1, policy_version 20770 (0.0007) -[2023-10-15 15:37:47,715][52866] Updated weights for policy 1, policy_version 20780 (0.0009) -[2023-10-15 15:37:48,077][52866] Updated weights for policy 1, policy_version 20790 (0.0009) -[2023-10-15 15:37:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 42500096. Throughput: 0: 1800.8, 1: 1795.6. Samples: 10632042. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-15 15:37:48,442][51532] Avg episode reward: [(0, '25.840'), (1, '24.970')] -[2023-10-15 15:37:48,446][52866] Updated weights for policy 1, policy_version 20800 (0.0010) -[2023-10-15 15:37:50,828][52833] Updated weights for policy 0, policy_version 20740 (0.0010) -[2023-10-15 15:37:51,188][52833] Updated weights for policy 0, policy_version 20750 (0.0011) -[2023-10-15 15:37:51,561][52833] Updated weights for policy 0, policy_version 20760 (0.0009) -[2023-10-15 15:37:52,212][52866] Updated weights for policy 1, policy_version 20810 (0.0008) -[2023-10-15 15:37:52,573][52866] Updated weights for policy 1, policy_version 20820 (0.0008) -[2023-10-15 15:37:52,939][52866] Updated weights for policy 1, policy_version 20830 (0.0009) -[2023-10-15 15:37:53,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 42598400. Throughput: 0: 1785.1, 1: 1807.6. Samples: 10652778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:37:53,442][51532] Avg episode reward: [(0, '25.550'), (1, '26.400')] -[2023-10-15 15:37:55,464][52833] Updated weights for policy 0, policy_version 20770 (0.0008) -[2023-10-15 15:37:55,840][52833] Updated weights for policy 0, policy_version 20780 (0.0007) -[2023-10-15 15:37:56,213][52833] Updated weights for policy 0, policy_version 20790 (0.0008) -[2023-10-15 15:37:56,577][52833] Updated weights for policy 0, policy_version 20800 (0.0010) -[2023-10-15 15:37:56,701][52866] Updated weights for policy 1, policy_version 20840 (0.0008) -[2023-10-15 15:37:57,075][52866] Updated weights for policy 1, policy_version 20850 (0.0008) -[2023-10-15 15:37:57,438][52866] Updated weights for policy 1, policy_version 20860 (0.0009) -[2023-10-15 15:37:58,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 42663936. Throughput: 0: 1782.5, 1: 1795.8. Samples: 10674004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:37:58,442][51532] Avg episode reward: [(0, '22.980'), (1, '27.960')] -[2023-10-15 15:38:00,246][52833] Updated weights for policy 0, policy_version 20810 (0.0007) -[2023-10-15 15:38:00,618][52833] Updated weights for policy 0, policy_version 20820 (0.0009) -[2023-10-15 15:38:00,994][52833] Updated weights for policy 0, policy_version 20830 (0.0010) -[2023-10-15 15:38:01,109][52866] Updated weights for policy 1, policy_version 20870 (0.0009) -[2023-10-15 15:38:01,472][52866] Updated weights for policy 1, policy_version 20880 (0.0009) -[2023-10-15 15:38:01,847][52866] Updated weights for policy 1, policy_version 20890 (0.0009) -[2023-10-15 15:38:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 42729472. Throughput: 0: 1787.5, 1: 1805.5. Samples: 10685358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:38:03,442][51532] Avg episode reward: [(0, '23.510'), (1, '26.580')] -[2023-10-15 15:38:04,658][52833] Updated weights for policy 0, policy_version 20840 (0.0010) -[2023-10-15 15:38:05,034][52833] Updated weights for policy 0, policy_version 20850 (0.0007) -[2023-10-15 15:38:05,398][52833] Updated weights for policy 0, policy_version 20860 (0.0007) -[2023-10-15 15:38:05,682][52866] Updated weights for policy 1, policy_version 20900 (0.0010) -[2023-10-15 15:38:06,052][52866] Updated weights for policy 1, policy_version 20910 (0.0007) -[2023-10-15 15:38:06,419][52866] Updated weights for policy 1, policy_version 20920 (0.0008) -[2023-10-15 15:38:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 42795008. Throughput: 0: 1782.2, 1: 1796.0. Samples: 10706262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:38:08,442][51532] Avg episode reward: [(0, '21.600'), (1, '28.890')] -[2023-10-15 15:38:09,280][52833] Updated weights for policy 0, policy_version 20870 (0.0008) -[2023-10-15 15:38:09,666][52833] Updated weights for policy 0, policy_version 20880 (0.0010) -[2023-10-15 15:38:10,022][52833] Updated weights for policy 0, policy_version 20890 (0.0010) -[2023-10-15 15:38:10,104][52866] Updated weights for policy 1, policy_version 20930 (0.0008) -[2023-10-15 15:38:10,494][52866] Updated weights for policy 1, policy_version 20940 (0.0007) -[2023-10-15 15:38:10,866][52866] Updated weights for policy 1, policy_version 20950 (0.0007) -[2023-10-15 15:38:11,220][52866] Updated weights for policy 1, policy_version 20960 (0.0007) -[2023-10-15 15:38:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 42860544. Throughput: 0: 1784.9, 1: 1797.6. Samples: 10728708. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) -[2023-10-15 15:38:13,441][51532] Avg episode reward: [(0, '23.340'), (1, '29.210')] -[2023-10-15 15:38:13,865][52833] Updated weights for policy 0, policy_version 20900 (0.0009) -[2023-10-15 15:38:14,237][52833] Updated weights for policy 0, policy_version 20910 (0.0007) -[2023-10-15 15:38:14,602][52833] Updated weights for policy 0, policy_version 20920 (0.0008) -[2023-10-15 15:38:14,978][52866] Updated weights for policy 1, policy_version 20970 (0.0008) -[2023-10-15 15:38:15,339][52866] Updated weights for policy 1, policy_version 20980 (0.0008) -[2023-10-15 15:38:15,704][52866] Updated weights for policy 1, policy_version 20990 (0.0008) -[2023-10-15 15:38:18,341][52833] Updated weights for policy 0, policy_version 20930 (0.0007) -[2023-10-15 15:38:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 42926080. Throughput: 0: 1783.1, 1: 1800.7. Samples: 10738696. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) -[2023-10-15 15:38:18,442][51532] Avg episode reward: [(0, '21.910'), (1, '28.690')] -[2023-10-15 15:38:18,716][52833] Updated weights for policy 0, policy_version 20940 (0.0008) -[2023-10-15 15:38:19,085][52833] Updated weights for policy 0, policy_version 20950 (0.0007) -[2023-10-15 15:38:19,339][52866] Updated weights for policy 1, policy_version 21000 (0.0009) -[2023-10-15 15:38:19,458][52833] Updated weights for policy 0, policy_version 20960 (0.0007) -[2023-10-15 15:38:19,710][52866] Updated weights for policy 1, policy_version 21010 (0.0008) -[2023-10-15 15:38:20,090][52866] Updated weights for policy 1, policy_version 21020 (0.0008) -[2023-10-15 15:38:23,226][52833] Updated weights for policy 0, policy_version 20970 (0.0008) -[2023-10-15 15:38:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 42991616. Throughput: 0: 1789.9, 1: 1799.8. Samples: 10761302. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) -[2023-10-15 15:38:23,442][51532] Avg episode reward: [(0, '24.310'), (1, '29.720')] -[2023-10-15 15:38:23,595][52833] Updated weights for policy 0, policy_version 20980 (0.0009) -[2023-10-15 15:38:23,865][52866] Updated weights for policy 1, policy_version 21030 (0.0008) -[2023-10-15 15:38:23,967][52833] Updated weights for policy 0, policy_version 20990 (0.0007) -[2023-10-15 15:38:24,236][52866] Updated weights for policy 1, policy_version 21040 (0.0009) -[2023-10-15 15:38:24,611][52866] Updated weights for policy 1, policy_version 21050 (0.0008) -[2023-10-15 15:38:27,708][52833] Updated weights for policy 0, policy_version 21000 (0.0010) -[2023-10-15 15:38:28,072][52833] Updated weights for policy 0, policy_version 21010 (0.0011) -[2023-10-15 15:38:28,395][52866] Updated weights for policy 1, policy_version 21060 (0.0008) -[2023-10-15 15:38:28,438][52833] Updated weights for policy 0, policy_version 21020 (0.0008) -[2023-10-15 15:38:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 43057152. Throughput: 0: 1809.0, 1: 1802.9. Samples: 10783098. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) -[2023-10-15 15:38:28,442][51532] Avg episode reward: [(0, '24.190'), (1, '30.130')] -[2023-10-15 15:38:28,763][52866] Updated weights for policy 1, policy_version 21070 (0.0008) -[2023-10-15 15:38:29,130][52866] Updated weights for policy 1, policy_version 21080 (0.0010) -[2023-10-15 15:38:32,110][52833] Updated weights for policy 0, policy_version 21030 (0.0007) -[2023-10-15 15:38:32,480][52833] Updated weights for policy 0, policy_version 21040 (0.0007) -[2023-10-15 15:38:32,850][52833] Updated weights for policy 0, policy_version 21050 (0.0007) -[2023-10-15 15:38:32,980][52866] Updated weights for policy 1, policy_version 21090 (0.0008) -[2023-10-15 15:38:33,336][52866] Updated weights for policy 1, policy_version 21100 (0.0007) -[2023-10-15 15:38:33,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 43155456. Throughput: 0: 1789.6, 1: 1797.3. Samples: 10793454. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-15 15:38:33,442][51532] Avg episode reward: [(0, '25.520'), (1, '32.140')] -[2023-10-15 15:38:33,698][52866] Updated weights for policy 1, policy_version 21110 (0.0008) -[2023-10-15 15:38:34,059][52518] Saving new best policy, reward=32.140! -[2023-10-15 15:38:34,063][52866] Updated weights for policy 1, policy_version 21120 (0.0008) -[2023-10-15 15:38:36,528][52833] Updated weights for policy 0, policy_version 21060 (0.0009) -[2023-10-15 15:38:36,900][52833] Updated weights for policy 0, policy_version 21070 (0.0009) -[2023-10-15 15:38:37,267][52833] Updated weights for policy 0, policy_version 21080 (0.0008) -[2023-10-15 15:38:37,861][52866] Updated weights for policy 1, policy_version 21130 (0.0007) -[2023-10-15 15:38:38,240][52866] Updated weights for policy 1, policy_version 21140 (0.0007) -[2023-10-15 15:38:38,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 43220992. Throughput: 0: 1812.3, 1: 1801.7. Samples: 10815408. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-15 15:38:38,441][51532] Avg episode reward: [(0, '27.380'), (1, '31.080')] -[2023-10-15 15:38:38,602][52866] Updated weights for policy 1, policy_version 21150 (0.0008) -[2023-10-15 15:38:40,988][52833] Updated weights for policy 0, policy_version 21090 (0.0008) -[2023-10-15 15:38:41,358][52833] Updated weights for policy 0, policy_version 21100 (0.0008) -[2023-10-15 15:38:41,729][52833] Updated weights for policy 0, policy_version 21110 (0.0010) -[2023-10-15 15:38:42,095][52833] Updated weights for policy 0, policy_version 21120 (0.0008) -[2023-10-15 15:38:42,324][52866] Updated weights for policy 1, policy_version 21160 (0.0007) -[2023-10-15 15:38:42,686][52866] Updated weights for policy 1, policy_version 21170 (0.0007) -[2023-10-15 15:38:43,063][52866] Updated weights for policy 1, policy_version 21180 (0.0008) -[2023-10-15 15:38:43,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 43319296. Throughput: 0: 1794.9, 1: 1804.4. Samples: 10835972. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-15 15:38:43,441][51532] Avg episode reward: [(0, '27.570'), (1, '29.500')] -[2023-10-15 15:38:45,776][52833] Updated weights for policy 0, policy_version 21130 (0.0008) -[2023-10-15 15:38:46,139][52833] Updated weights for policy 0, policy_version 21140 (0.0008) -[2023-10-15 15:38:46,506][52833] Updated weights for policy 0, policy_version 21150 (0.0010) -[2023-10-15 15:38:46,808][52866] Updated weights for policy 1, policy_version 21190 (0.0009) -[2023-10-15 15:38:47,179][52866] Updated weights for policy 1, policy_version 21200 (0.0009) -[2023-10-15 15:38:47,530][52866] Updated weights for policy 1, policy_version 21210 (0.0009) -[2023-10-15 15:38:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 43384832. Throughput: 0: 1810.2, 1: 1800.9. Samples: 10847856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:38:48,441][51532] Avg episode reward: [(0, '27.750'), (1, '27.260')] -[2023-10-15 15:38:50,360][52833] Updated weights for policy 0, policy_version 21160 (0.0007) -[2023-10-15 15:38:50,733][52833] Updated weights for policy 0, policy_version 21170 (0.0008) -[2023-10-15 15:38:51,099][52833] Updated weights for policy 0, policy_version 21180 (0.0009) -[2023-10-15 15:38:51,115][52866] Updated weights for policy 1, policy_version 21220 (0.0008) -[2023-10-15 15:38:51,484][52866] Updated weights for policy 1, policy_version 21230 (0.0007) -[2023-10-15 15:38:51,862][52866] Updated weights for policy 1, policy_version 21240 (0.0008) -[2023-10-15 15:38:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43450368. Throughput: 0: 1793.2, 1: 1808.9. Samples: 10868356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:38:53,442][51532] Avg episode reward: [(0, '26.770'), (1, '29.700')] -[2023-10-15 15:38:54,933][52833] Updated weights for policy 0, policy_version 21190 (0.0009) -[2023-10-15 15:38:55,319][52833] Updated weights for policy 0, policy_version 21200 (0.0007) -[2023-10-15 15:38:55,602][52866] Updated weights for policy 1, policy_version 21250 (0.0007) -[2023-10-15 15:38:55,683][52833] Updated weights for policy 0, policy_version 21210 (0.0007) -[2023-10-15 15:38:55,986][52866] Updated weights for policy 1, policy_version 21260 (0.0007) -[2023-10-15 15:38:56,363][52866] Updated weights for policy 1, policy_version 21270 (0.0009) -[2023-10-15 15:38:56,728][52866] Updated weights for policy 1, policy_version 21280 (0.0008) -[2023-10-15 15:38:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43515904. Throughput: 0: 1796.1, 1: 1796.2. Samples: 10890362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:38:58,441][51532] Avg episode reward: [(0, '28.820'), (1, '29.410')] -[2023-10-15 15:38:59,418][52833] Updated weights for policy 0, policy_version 21220 (0.0007) -[2023-10-15 15:38:59,789][52833] Updated weights for policy 0, policy_version 21230 (0.0009) -[2023-10-15 15:39:00,168][52833] Updated weights for policy 0, policy_version 21240 (0.0008) -[2023-10-15 15:39:00,356][52866] Updated weights for policy 1, policy_version 21290 (0.0009) -[2023-10-15 15:39:00,726][52866] Updated weights for policy 1, policy_version 21300 (0.0007) -[2023-10-15 15:39:01,101][52866] Updated weights for policy 1, policy_version 21310 (0.0010) -[2023-10-15 15:39:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 43581440. Throughput: 0: 1791.5, 1: 1801.4. Samples: 10900378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:39:03,441][51532] Avg episode reward: [(0, '27.750'), (1, '29.440')] -[2023-10-15 15:39:03,958][52833] Updated weights for policy 0, policy_version 21250 (0.0009) -[2023-10-15 15:39:04,322][52833] Updated weights for policy 0, policy_version 21260 (0.0010) -[2023-10-15 15:39:04,689][52833] Updated weights for policy 0, policy_version 21270 (0.0008) -[2023-10-15 15:39:04,779][52866] Updated weights for policy 1, policy_version 21320 (0.0008) -[2023-10-15 15:39:05,057][52833] Updated weights for policy 0, policy_version 21280 (0.0008) -[2023-10-15 15:39:05,146][52866] Updated weights for policy 1, policy_version 21330 (0.0008) -[2023-10-15 15:39:05,515][52866] Updated weights for policy 1, policy_version 21340 (0.0009) -[2023-10-15 15:39:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 43646976. Throughput: 0: 1784.6, 1: 1795.4. Samples: 10922400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:39:08,441][51532] Avg episode reward: [(0, '28.590'), (1, '30.200')] -[2023-10-15 15:39:08,779][52833] Updated weights for policy 0, policy_version 21290 (0.0009) -[2023-10-15 15:39:09,151][52833] Updated weights for policy 0, policy_version 21300 (0.0009) -[2023-10-15 15:39:09,420][52866] Updated weights for policy 1, policy_version 21350 (0.0009) -[2023-10-15 15:39:09,518][52833] Updated weights for policy 0, policy_version 21310 (0.0008) -[2023-10-15 15:39:09,789][52866] Updated weights for policy 1, policy_version 21360 (0.0010) -[2023-10-15 15:39:10,155][52866] Updated weights for policy 1, policy_version 21370 (0.0009) -[2023-10-15 15:39:13,105][52833] Updated weights for policy 0, policy_version 21320 (0.0007) -[2023-10-15 15:39:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 43712512. Throughput: 0: 1795.2, 1: 1795.8. Samples: 10944692. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:39:13,441][51532] Avg episode reward: [(0, '26.970'), (1, '27.990')] -[2023-10-15 15:39:13,476][52833] Updated weights for policy 0, policy_version 21330 (0.0007) -[2023-10-15 15:39:13,841][52833] Updated weights for policy 0, policy_version 21340 (0.0008) -[2023-10-15 15:39:13,871][52866] Updated weights for policy 1, policy_version 21380 (0.0008) -[2023-10-15 15:39:14,233][52866] Updated weights for policy 1, policy_version 21390 (0.0008) -[2023-10-15 15:39:14,603][52866] Updated weights for policy 1, policy_version 21400 (0.0008) -[2023-10-15 15:39:17,840][52833] Updated weights for policy 0, policy_version 21350 (0.0008) -[2023-10-15 15:39:18,206][52833] Updated weights for policy 0, policy_version 21360 (0.0007) -[2023-10-15 15:39:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 43778048. Throughput: 0: 1781.5, 1: 1795.0. Samples: 10954396. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 15:39:18,442][51532] Avg episode reward: [(0, '26.960'), (1, '30.370')] -[2023-10-15 15:39:18,574][52833] Updated weights for policy 0, policy_version 21370 (0.0009) -[2023-10-15 15:39:18,623][52866] Updated weights for policy 1, policy_version 21410 (0.0009) -[2023-10-15 15:39:18,992][52866] Updated weights for policy 1, policy_version 21420 (0.0007) -[2023-10-15 15:39:19,357][52866] Updated weights for policy 1, policy_version 21430 (0.0008) -[2023-10-15 15:39:19,728][52866] Updated weights for policy 1, policy_version 21440 (0.0007) -[2023-10-15 15:39:22,370][52833] Updated weights for policy 0, policy_version 21380 (0.0009) -[2023-10-15 15:39:22,738][52833] Updated weights for policy 0, policy_version 21390 (0.0008) -[2023-10-15 15:39:23,100][52833] Updated weights for policy 0, policy_version 21400 (0.0008) -[2023-10-15 15:39:23,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 43876352. Throughput: 0: 1788.5, 1: 1790.9. Samples: 10976482. Policy #0 lag: (min: 12.0, avg: 19.7, max: 44.0) -[2023-10-15 15:39:23,441][51532] Avg episode reward: [(0, '27.030'), (1, '30.790')] -[2023-10-15 15:39:23,473][52866] Updated weights for policy 1, policy_version 21450 (0.0008) -[2023-10-15 15:39:23,841][52866] Updated weights for policy 1, policy_version 21460 (0.0008) -[2023-10-15 15:39:24,209][52866] Updated weights for policy 1, policy_version 21470 (0.0011) -[2023-10-15 15:39:26,943][52833] Updated weights for policy 0, policy_version 21410 (0.0007) -[2023-10-15 15:39:27,320][52833] Updated weights for policy 0, policy_version 21420 (0.0007) -[2023-10-15 15:39:27,688][52833] Updated weights for policy 0, policy_version 21430 (0.0009) -[2023-10-15 15:39:27,770][52866] Updated weights for policy 1, policy_version 21480 (0.0008) -[2023-10-15 15:39:28,053][52833] Updated weights for policy 0, policy_version 21440 (0.0009) -[2023-10-15 15:39:28,138][52866] Updated weights for policy 1, policy_version 21490 (0.0007) -[2023-10-15 15:39:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 43941888. Throughput: 0: 1778.1, 1: 1810.0. Samples: 10997438. Policy #0 lag: (min: 12.0, avg: 19.7, max: 44.0) -[2023-10-15 15:39:28,442][51532] Avg episode reward: [(0, '26.460'), (1, '33.150')] -[2023-10-15 15:39:28,505][52866] Updated weights for policy 1, policy_version 21500 (0.0007) -[2023-10-15 15:39:28,644][52518] Saving new best policy, reward=33.150! -[2023-10-15 15:39:31,840][52833] Updated weights for policy 0, policy_version 21450 (0.0007) -[2023-10-15 15:39:32,211][52833] Updated weights for policy 0, policy_version 21460 (0.0007) -[2023-10-15 15:39:32,389][52866] Updated weights for policy 1, policy_version 21510 (0.0010) -[2023-10-15 15:39:32,581][52833] Updated weights for policy 0, policy_version 21470 (0.0008) -[2023-10-15 15:39:32,762][52866] Updated weights for policy 1, policy_version 21520 (0.0007) -[2023-10-15 15:39:33,127][52866] Updated weights for policy 1, policy_version 21530 (0.0007) -[2023-10-15 15:39:33,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 44040192. Throughput: 0: 1785.2, 1: 1792.8. Samples: 11008866. Policy #0 lag: (min: 12.0, avg: 19.7, max: 44.0) -[2023-10-15 15:39:33,442][51532] Avg episode reward: [(0, '26.870'), (1, '32.050')] -[2023-10-15 15:39:36,389][52833] Updated weights for policy 0, policy_version 21480 (0.0008) -[2023-10-15 15:39:36,750][52833] Updated weights for policy 0, policy_version 21490 (0.0008) -[2023-10-15 15:39:36,775][52866] Updated weights for policy 1, policy_version 21540 (0.0008) -[2023-10-15 15:39:37,122][52833] Updated weights for policy 0, policy_version 21500 (0.0008) -[2023-10-15 15:39:37,141][52866] Updated weights for policy 1, policy_version 21550 (0.0008) -[2023-10-15 15:39:37,513][52866] Updated weights for policy 1, policy_version 21560 (0.0010) -[2023-10-15 15:39:38,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 44105728. Throughput: 0: 1784.6, 1: 1811.4. Samples: 11030178. Policy #0 lag: (min: 12.0, avg: 19.7, max: 44.0) -[2023-10-15 15:39:38,441][51532] Avg episode reward: [(0, '26.960'), (1, '32.470')] -[2023-10-15 15:39:40,916][52833] Updated weights for policy 0, policy_version 21510 (0.0008) -[2023-10-15 15:39:41,165][52866] Updated weights for policy 1, policy_version 21570 (0.0010) -[2023-10-15 15:39:41,297][52833] Updated weights for policy 0, policy_version 21520 (0.0010) -[2023-10-15 15:39:41,547][52866] Updated weights for policy 1, policy_version 21580 (0.0008) -[2023-10-15 15:39:41,658][52833] Updated weights for policy 0, policy_version 21530 (0.0008) -[2023-10-15 15:39:41,912][52866] Updated weights for policy 1, policy_version 21590 (0.0009) -[2023-10-15 15:39:42,284][52866] Updated weights for policy 1, policy_version 21600 (0.0007) -[2023-10-15 15:39:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 44171264. Throughput: 0: 1774.1, 1: 1795.0. Samples: 11050972. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-15 15:39:43,442][51532] Avg episode reward: [(0, '28.160'), (1, '33.430')] -[2023-10-15 15:39:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000021600_22118400.pth... -[2023-10-15 15:39:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000021536_22052864.pth... -[2023-10-15 15:39:43,487][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000019872_20348928.pth -[2023-10-15 15:39:43,492][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000019904_20381696.pth -[2023-10-15 15:39:43,496][52518] Saving new best policy, reward=33.430! -[2023-10-15 15:39:45,149][52833] Updated weights for policy 0, policy_version 21540 (0.0007) -[2023-10-15 15:39:45,524][52833] Updated weights for policy 0, policy_version 21550 (0.0009) -[2023-10-15 15:39:45,886][52833] Updated weights for policy 0, policy_version 21560 (0.0009) -[2023-10-15 15:39:45,927][52866] Updated weights for policy 1, policy_version 21610 (0.0008) -[2023-10-15 15:39:46,293][52866] Updated weights for policy 1, policy_version 21620 (0.0007) -[2023-10-15 15:39:46,664][52866] Updated weights for policy 1, policy_version 21630 (0.0007) -[2023-10-15 15:39:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 44236800. Throughput: 0: 1790.8, 1: 1808.9. Samples: 11062368. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-15 15:39:48,441][51532] Avg episode reward: [(0, '26.690'), (1, '32.340')] -[2023-10-15 15:39:49,806][52833] Updated weights for policy 0, policy_version 21570 (0.0010) -[2023-10-15 15:39:50,177][52833] Updated weights for policy 0, policy_version 21580 (0.0008) -[2023-10-15 15:39:50,354][52866] Updated weights for policy 1, policy_version 21640 (0.0008) -[2023-10-15 15:39:50,543][52833] Updated weights for policy 0, policy_version 21590 (0.0007) -[2023-10-15 15:39:50,720][52866] Updated weights for policy 1, policy_version 21650 (0.0007) -[2023-10-15 15:39:50,910][52833] Updated weights for policy 0, policy_version 21600 (0.0009) -[2023-10-15 15:39:51,092][52866] Updated weights for policy 1, policy_version 21660 (0.0010) -[2023-10-15 15:39:53,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 44302336. Throughput: 0: 1777.6, 1: 1793.2. Samples: 11083086. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-15 15:39:53,442][51532] Avg episode reward: [(0, '27.040'), (1, '33.180')] -[2023-10-15 15:39:54,691][52833] Updated weights for policy 0, policy_version 21610 (0.0009) -[2023-10-15 15:39:54,813][52866] Updated weights for policy 1, policy_version 21670 (0.0011) -[2023-10-15 15:39:55,062][52833] Updated weights for policy 0, policy_version 21620 (0.0008) -[2023-10-15 15:39:55,190][52866] Updated weights for policy 1, policy_version 21680 (0.0008) -[2023-10-15 15:39:55,430][52833] Updated weights for policy 0, policy_version 21630 (0.0008) -[2023-10-15 15:39:55,549][52866] Updated weights for policy 1, policy_version 21690 (0.0007) -[2023-10-15 15:39:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 44367872. Throughput: 0: 1777.5, 1: 1796.4. Samples: 11105520. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-15 15:39:58,442][51532] Avg episode reward: [(0, '28.920'), (1, '32.040')] -[2023-10-15 15:39:59,222][52833] Updated weights for policy 0, policy_version 21640 (0.0010) -[2023-10-15 15:39:59,253][52866] Updated weights for policy 1, policy_version 21700 (0.0009) -[2023-10-15 15:39:59,591][52833] Updated weights for policy 0, policy_version 21650 (0.0008) -[2023-10-15 15:39:59,615][52866] Updated weights for policy 1, policy_version 21710 (0.0008) -[2023-10-15 15:39:59,945][52833] Updated weights for policy 0, policy_version 21660 (0.0008) -[2023-10-15 15:39:59,973][52866] Updated weights for policy 1, policy_version 21720 (0.0008) -[2023-10-15 15:40:03,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 44433408. Throughput: 0: 1774.3, 1: 1798.7. Samples: 11115182. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:40:03,441][51532] Avg episode reward: [(0, '28.770'), (1, '30.970')] -[2023-10-15 15:40:03,782][52866] Updated weights for policy 1, policy_version 21730 (0.0010) -[2023-10-15 15:40:03,823][52833] Updated weights for policy 0, policy_version 21670 (0.0008) -[2023-10-15 15:40:04,146][52866] Updated weights for policy 1, policy_version 21740 (0.0010) -[2023-10-15 15:40:04,185][52833] Updated weights for policy 0, policy_version 21680 (0.0008) -[2023-10-15 15:40:04,506][52866] Updated weights for policy 1, policy_version 21750 (0.0007) -[2023-10-15 15:40:04,558][52833] Updated weights for policy 0, policy_version 21690 (0.0008) -[2023-10-15 15:40:04,867][52866] Updated weights for policy 1, policy_version 21760 (0.0008) -[2023-10-15 15:40:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 44498944. Throughput: 0: 1767.2, 1: 1797.9. Samples: 11136910. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:40:08,441][51532] Avg episode reward: [(0, '29.170'), (1, '28.950')] -[2023-10-15 15:40:08,473][52833] Updated weights for policy 0, policy_version 21700 (0.0008) -[2023-10-15 15:40:08,655][52866] Updated weights for policy 1, policy_version 21770 (0.0009) -[2023-10-15 15:40:08,842][52833] Updated weights for policy 0, policy_version 21710 (0.0007) -[2023-10-15 15:40:09,016][52866] Updated weights for policy 1, policy_version 21780 (0.0009) -[2023-10-15 15:40:09,210][52833] Updated weights for policy 0, policy_version 21720 (0.0007) -[2023-10-15 15:40:09,393][52866] Updated weights for policy 1, policy_version 21790 (0.0009) -[2023-10-15 15:40:09,497][52410] Saving new best policy, reward=29.170! -[2023-10-15 15:40:13,123][52833] Updated weights for policy 0, policy_version 21730 (0.0007) -[2023-10-15 15:40:13,222][52866] Updated weights for policy 1, policy_version 21800 (0.0008) -[2023-10-15 15:40:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 44564480. Throughput: 0: 1792.7, 1: 1801.5. Samples: 11159176. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:40:13,442][51532] Avg episode reward: [(0, '28.820'), (1, '29.120')] -[2023-10-15 15:40:13,489][52833] Updated weights for policy 0, policy_version 21740 (0.0010) -[2023-10-15 15:40:13,582][52866] Updated weights for policy 1, policy_version 21810 (0.0008) -[2023-10-15 15:40:13,859][52833] Updated weights for policy 0, policy_version 21750 (0.0008) -[2023-10-15 15:40:13,948][52866] Updated weights for policy 1, policy_version 21820 (0.0009) -[2023-10-15 15:40:14,219][52833] Updated weights for policy 0, policy_version 21760 (0.0010) -[2023-10-15 15:40:17,729][52866] Updated weights for policy 1, policy_version 21830 (0.0008) -[2023-10-15 15:40:17,828][52833] Updated weights for policy 0, policy_version 21770 (0.0009) -[2023-10-15 15:40:18,100][52866] Updated weights for policy 1, policy_version 21840 (0.0008) -[2023-10-15 15:40:18,198][52833] Updated weights for policy 0, policy_version 21780 (0.0009) -[2023-10-15 15:40:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 44630016. Throughput: 0: 1767.3, 1: 1793.6. Samples: 11169108. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:40:18,442][51532] Avg episode reward: [(0, '27.660'), (1, '29.610')] -[2023-10-15 15:40:18,465][52866] Updated weights for policy 1, policy_version 21850 (0.0009) -[2023-10-15 15:40:18,572][52833] Updated weights for policy 0, policy_version 21790 (0.0008) -[2023-10-15 15:40:22,203][52866] Updated weights for policy 1, policy_version 21860 (0.0007) -[2023-10-15 15:40:22,488][52833] Updated weights for policy 0, policy_version 21800 (0.0009) -[2023-10-15 15:40:22,571][52866] Updated weights for policy 1, policy_version 21870 (0.0007) -[2023-10-15 15:40:22,865][52833] Updated weights for policy 0, policy_version 21810 (0.0008) -[2023-10-15 15:40:22,937][52866] Updated weights for policy 1, policy_version 21880 (0.0007) -[2023-10-15 15:40:23,223][52833] Updated weights for policy 0, policy_version 21820 (0.0007) -[2023-10-15 15:40:23,441][51532] Fps is (10 sec: 19661.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 44761088. Throughput: 0: 1784.4, 1: 1799.0. Samples: 11191430. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) -[2023-10-15 15:40:23,442][51532] Avg episode reward: [(0, '28.840'), (1, '29.730')] -[2023-10-15 15:40:26,675][52866] Updated weights for policy 1, policy_version 21890 (0.0007) -[2023-10-15 15:40:27,044][52866] Updated weights for policy 1, policy_version 21900 (0.0007) -[2023-10-15 15:40:27,069][52833] Updated weights for policy 0, policy_version 21830 (0.0009) -[2023-10-15 15:40:27,410][52866] Updated weights for policy 1, policy_version 21910 (0.0008) -[2023-10-15 15:40:27,445][52833] Updated weights for policy 0, policy_version 21840 (0.0008) -[2023-10-15 15:40:27,775][52866] Updated weights for policy 1, policy_version 21920 (0.0008) -[2023-10-15 15:40:27,812][52833] Updated weights for policy 0, policy_version 21850 (0.0008) -[2023-10-15 15:40:28,441][51532] Fps is (10 sec: 19661.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 44826624. Throughput: 0: 1771.9, 1: 1790.8. Samples: 11211292. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) -[2023-10-15 15:40:28,441][51532] Avg episode reward: [(0, '28.720'), (1, '28.980')] -[2023-10-15 15:40:31,544][52866] Updated weights for policy 1, policy_version 21930 (0.0009) -[2023-10-15 15:40:31,696][52833] Updated weights for policy 0, policy_version 21860 (0.0009) -[2023-10-15 15:40:31,907][52866] Updated weights for policy 1, policy_version 21940 (0.0009) -[2023-10-15 15:40:32,063][52833] Updated weights for policy 0, policy_version 21870 (0.0008) -[2023-10-15 15:40:32,272][52866] Updated weights for policy 1, policy_version 21950 (0.0007) -[2023-10-15 15:40:32,430][52833] Updated weights for policy 0, policy_version 21880 (0.0007) -[2023-10-15 15:40:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 44892160. Throughput: 0: 1781.5, 1: 1802.2. Samples: 11223636. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) -[2023-10-15 15:40:33,441][51532] Avg episode reward: [(0, '26.130'), (1, '29.760')] -[2023-10-15 15:40:36,061][52866] Updated weights for policy 1, policy_version 21960 (0.0009) -[2023-10-15 15:40:36,176][52833] Updated weights for policy 0, policy_version 21890 (0.0008) -[2023-10-15 15:40:36,426][52866] Updated weights for policy 1, policy_version 21970 (0.0009) -[2023-10-15 15:40:36,550][52833] Updated weights for policy 0, policy_version 21900 (0.0008) -[2023-10-15 15:40:36,787][52866] Updated weights for policy 1, policy_version 21980 (0.0008) -[2023-10-15 15:40:36,923][52833] Updated weights for policy 0, policy_version 21910 (0.0008) -[2023-10-15 15:40:37,295][52833] Updated weights for policy 0, policy_version 21920 (0.0008) -[2023-10-15 15:40:38,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 44957696. Throughput: 0: 1777.5, 1: 1790.2. Samples: 11243634. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) -[2023-10-15 15:40:38,442][51532] Avg episode reward: [(0, '29.210'), (1, '28.980')] -[2023-10-15 15:40:38,443][52410] Saving new best policy, reward=29.210! -[2023-10-15 15:40:40,576][52866] Updated weights for policy 1, policy_version 21990 (0.0008) -[2023-10-15 15:40:40,942][52866] Updated weights for policy 1, policy_version 22000 (0.0008) -[2023-10-15 15:40:40,946][52833] Updated weights for policy 0, policy_version 21930 (0.0007) -[2023-10-15 15:40:41,311][52866] Updated weights for policy 1, policy_version 22010 (0.0010) -[2023-10-15 15:40:41,321][52833] Updated weights for policy 0, policy_version 21940 (0.0010) -[2023-10-15 15:40:41,682][52833] Updated weights for policy 0, policy_version 21950 (0.0008) -[2023-10-15 15:40:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45023232. Throughput: 0: 1763.5, 1: 1789.2. Samples: 11265390. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) -[2023-10-15 15:40:43,442][51532] Avg episode reward: [(0, '27.770'), (1, '31.380')] -[2023-10-15 15:40:45,095][52866] Updated weights for policy 1, policy_version 22020 (0.0009) -[2023-10-15 15:40:45,465][52866] Updated weights for policy 1, policy_version 22030 (0.0009) -[2023-10-15 15:40:45,542][52833] Updated weights for policy 0, policy_version 21960 (0.0008) -[2023-10-15 15:40:45,826][52866] Updated weights for policy 1, policy_version 22040 (0.0008) -[2023-10-15 15:40:45,915][52833] Updated weights for policy 0, policy_version 21970 (0.0008) -[2023-10-15 15:40:46,288][52833] Updated weights for policy 0, policy_version 21980 (0.0008) -[2023-10-15 15:40:48,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 45088768. Throughput: 0: 1780.3, 1: 1794.6. Samples: 11276052. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) -[2023-10-15 15:40:48,442][51532] Avg episode reward: [(0, '28.290'), (1, '31.160')] -[2023-10-15 15:40:49,580][52866] Updated weights for policy 1, policy_version 22050 (0.0009) -[2023-10-15 15:40:49,935][52866] Updated weights for policy 1, policy_version 22060 (0.0008) -[2023-10-15 15:40:50,074][52833] Updated weights for policy 0, policy_version 21990 (0.0008) -[2023-10-15 15:40:50,301][52866] Updated weights for policy 1, policy_version 22070 (0.0007) -[2023-10-15 15:40:50,457][52833] Updated weights for policy 0, policy_version 22000 (0.0008) -[2023-10-15 15:40:50,670][52866] Updated weights for policy 1, policy_version 22080 (0.0010) -[2023-10-15 15:40:50,828][52833] Updated weights for policy 0, policy_version 22010 (0.0009) -[2023-10-15 15:40:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 45154304. Throughput: 0: 1774.6, 1: 1788.1. Samples: 11297232. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) -[2023-10-15 15:40:53,442][51532] Avg episode reward: [(0, '27.920'), (1, '31.380')] -[2023-10-15 15:40:54,407][52866] Updated weights for policy 1, policy_version 22090 (0.0009) -[2023-10-15 15:40:54,512][52833] Updated weights for policy 0, policy_version 22020 (0.0008) -[2023-10-15 15:40:54,769][52866] Updated weights for policy 1, policy_version 22100 (0.0010) -[2023-10-15 15:40:54,878][52833] Updated weights for policy 0, policy_version 22030 (0.0009) -[2023-10-15 15:40:55,126][52866] Updated weights for policy 1, policy_version 22110 (0.0008) -[2023-10-15 15:40:55,254][52833] Updated weights for policy 0, policy_version 22040 (0.0008) -[2023-10-15 15:40:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 45219840. Throughput: 0: 1774.6, 1: 1790.7. Samples: 11319614. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 15:40:58,441][51532] Avg episode reward: [(0, '26.630'), (1, '33.720')] -[2023-10-15 15:40:58,752][52866] Updated weights for policy 1, policy_version 22120 (0.0010) -[2023-10-15 15:40:59,016][52833] Updated weights for policy 0, policy_version 22050 (0.0009) -[2023-10-15 15:40:59,116][52866] Updated weights for policy 1, policy_version 22130 (0.0010) -[2023-10-15 15:40:59,382][52833] Updated weights for policy 0, policy_version 22060 (0.0008) -[2023-10-15 15:40:59,482][52866] Updated weights for policy 1, policy_version 22140 (0.0007) -[2023-10-15 15:40:59,618][52518] Saving new best policy, reward=33.720! -[2023-10-15 15:40:59,742][52833] Updated weights for policy 0, policy_version 22070 (0.0008) -[2023-10-15 15:41:00,105][52833] Updated weights for policy 0, policy_version 22080 (0.0012) -[2023-10-15 15:41:03,305][52866] Updated weights for policy 1, policy_version 22150 (0.0009) -[2023-10-15 15:41:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 45285376. Throughput: 0: 1771.9, 1: 1789.9. Samples: 11329390. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 15:41:03,442][51532] Avg episode reward: [(0, '28.420'), (1, '32.110')] -[2023-10-15 15:41:03,681][52866] Updated weights for policy 1, policy_version 22160 (0.0009) -[2023-10-15 15:41:03,979][52833] Updated weights for policy 0, policy_version 22090 (0.0008) -[2023-10-15 15:41:04,042][52866] Updated weights for policy 1, policy_version 22170 (0.0008) -[2023-10-15 15:41:04,347][52833] Updated weights for policy 0, policy_version 22100 (0.0008) -[2023-10-15 15:41:04,712][52833] Updated weights for policy 0, policy_version 22110 (0.0008) -[2023-10-15 15:41:07,744][52866] Updated weights for policy 1, policy_version 22180 (0.0008) -[2023-10-15 15:41:08,116][52866] Updated weights for policy 1, policy_version 22190 (0.0007) -[2023-10-15 15:41:08,404][52833] Updated weights for policy 0, policy_version 22120 (0.0008) -[2023-10-15 15:41:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 45350912. Throughput: 0: 1774.6, 1: 1790.1. Samples: 11351842. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 15:41:08,441][51532] Avg episode reward: [(0, '28.770'), (1, '33.730')] -[2023-10-15 15:41:08,479][52866] Updated weights for policy 1, policy_version 22200 (0.0008) -[2023-10-15 15:41:08,761][52518] Saving new best policy, reward=33.730! -[2023-10-15 15:41:08,771][52833] Updated weights for policy 0, policy_version 22130 (0.0008) -[2023-10-15 15:41:09,148][52833] Updated weights for policy 0, policy_version 22140 (0.0007) -[2023-10-15 15:41:12,238][52866] Updated weights for policy 1, policy_version 22210 (0.0010) -[2023-10-15 15:41:12,630][52866] Updated weights for policy 1, policy_version 22220 (0.0008) -[2023-10-15 15:41:12,995][52866] Updated weights for policy 1, policy_version 22230 (0.0008) -[2023-10-15 15:41:13,010][52833] Updated weights for policy 0, policy_version 22150 (0.0007) -[2023-10-15 15:41:13,353][52866] Updated weights for policy 1, policy_version 22240 (0.0009) -[2023-10-15 15:41:13,371][52833] Updated weights for policy 0, policy_version 22160 (0.0008) -[2023-10-15 15:41:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 45449216. Throughput: 0: 1797.2, 1: 1798.4. Samples: 11373094. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 15:41:13,442][51532] Avg episode reward: [(0, '28.460'), (1, '33.770')] -[2023-10-15 15:41:13,456][52518] Saving new best policy, reward=33.770! -[2023-10-15 15:41:13,747][52833] Updated weights for policy 0, policy_version 22170 (0.0009) -[2023-10-15 15:41:17,077][52866] Updated weights for policy 1, policy_version 22250 (0.0008) -[2023-10-15 15:41:17,443][52866] Updated weights for policy 1, policy_version 22260 (0.0007) -[2023-10-15 15:41:17,468][52833] Updated weights for policy 0, policy_version 22180 (0.0009) -[2023-10-15 15:41:17,810][52866] Updated weights for policy 1, policy_version 22270 (0.0008) -[2023-10-15 15:41:17,835][52833] Updated weights for policy 0, policy_version 22190 (0.0008) -[2023-10-15 15:41:18,203][52833] Updated weights for policy 0, policy_version 22200 (0.0009) -[2023-10-15 15:41:18,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 45514752. Throughput: 0: 1776.3, 1: 1784.2. Samples: 11383862. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 15:41:18,442][51532] Avg episode reward: [(0, '29.980'), (1, '33.150')] -[2023-10-15 15:41:18,499][52410] Saving new best policy, reward=29.980! -[2023-10-15 15:41:21,645][52866] Updated weights for policy 1, policy_version 22280 (0.0008) -[2023-10-15 15:41:21,971][52833] Updated weights for policy 0, policy_version 22210 (0.0010) -[2023-10-15 15:41:22,005][52866] Updated weights for policy 1, policy_version 22290 (0.0007) -[2023-10-15 15:41:22,339][52833] Updated weights for policy 0, policy_version 22220 (0.0007) -[2023-10-15 15:41:22,364][52866] Updated weights for policy 1, policy_version 22300 (0.0007) -[2023-10-15 15:41:22,709][52833] Updated weights for policy 0, policy_version 22230 (0.0008) -[2023-10-15 15:41:23,078][52833] Updated weights for policy 0, policy_version 22240 (0.0007) -[2023-10-15 15:41:23,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 45613056. Throughput: 0: 1794.4, 1: 1798.0. Samples: 11405296. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 15:41:23,442][51532] Avg episode reward: [(0, '30.150'), (1, '31.380')] -[2023-10-15 15:41:23,442][52410] Saving new best policy, reward=30.150! -[2023-10-15 15:41:26,054][52866] Updated weights for policy 1, policy_version 22310 (0.0009) -[2023-10-15 15:41:26,420][52866] Updated weights for policy 1, policy_version 22320 (0.0008) -[2023-10-15 15:41:26,786][52866] Updated weights for policy 1, policy_version 22330 (0.0009) -[2023-10-15 15:41:26,856][52833] Updated weights for policy 0, policy_version 22250 (0.0009) -[2023-10-15 15:41:27,229][52833] Updated weights for policy 0, policy_version 22260 (0.0007) -[2023-10-15 15:41:27,599][52833] Updated weights for policy 0, policy_version 22270 (0.0008) -[2023-10-15 15:41:28,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45678592. Throughput: 0: 1780.0, 1: 1785.1. Samples: 11425818. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 15:41:28,441][51532] Avg episode reward: [(0, '32.290'), (1, '30.520')] -[2023-10-15 15:41:28,449][52410] Saving new best policy, reward=32.290! -[2023-10-15 15:41:30,550][52866] Updated weights for policy 1, policy_version 22340 (0.0008) -[2023-10-15 15:41:30,922][52866] Updated weights for policy 1, policy_version 22350 (0.0007) -[2023-10-15 15:41:31,288][52866] Updated weights for policy 1, policy_version 22360 (0.0010) -[2023-10-15 15:41:31,338][52833] Updated weights for policy 0, policy_version 22280 (0.0009) -[2023-10-15 15:41:31,702][52833] Updated weights for policy 0, policy_version 22290 (0.0008) -[2023-10-15 15:41:32,079][52833] Updated weights for policy 0, policy_version 22300 (0.0009) -[2023-10-15 15:41:33,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45744128. Throughput: 0: 1800.2, 1: 1793.8. Samples: 11437782. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 15:41:33,441][51532] Avg episode reward: [(0, '32.620'), (1, '31.220')] -[2023-10-15 15:41:33,442][52410] Saving new best policy, reward=32.620! -[2023-10-15 15:41:35,121][52866] Updated weights for policy 1, policy_version 22370 (0.0008) -[2023-10-15 15:41:35,492][52866] Updated weights for policy 1, policy_version 22380 (0.0007) -[2023-10-15 15:41:35,821][52833] Updated weights for policy 0, policy_version 22310 (0.0015) -[2023-10-15 15:41:35,860][52866] Updated weights for policy 1, policy_version 22390 (0.0007) -[2023-10-15 15:41:36,186][52833] Updated weights for policy 0, policy_version 22320 (0.0008) -[2023-10-15 15:41:36,232][52866] Updated weights for policy 1, policy_version 22400 (0.0008) -[2023-10-15 15:41:36,558][52833] Updated weights for policy 0, policy_version 22330 (0.0010) -[2023-10-15 15:41:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 45809664. Throughput: 0: 1781.8, 1: 1790.3. Samples: 11457978. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 15:41:38,441][51532] Avg episode reward: [(0, '32.310'), (1, '30.060')] -[2023-10-15 15:41:39,893][52866] Updated weights for policy 1, policy_version 22410 (0.0011) -[2023-10-15 15:41:40,229][52833] Updated weights for policy 0, policy_version 22340 (0.0009) -[2023-10-15 15:41:40,261][52866] Updated weights for policy 1, policy_version 22420 (0.0009) -[2023-10-15 15:41:40,605][52833] Updated weights for policy 0, policy_version 22350 (0.0008) -[2023-10-15 15:41:40,630][52866] Updated weights for policy 1, policy_version 22430 (0.0008) -[2023-10-15 15:41:40,971][52833] Updated weights for policy 0, policy_version 22360 (0.0009) -[2023-10-15 15:41:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 45875200. Throughput: 0: 1779.5, 1: 1790.3. Samples: 11480252. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 15:41:43,442][51532] Avg episode reward: [(0, '32.970'), (1, '29.210')] -[2023-10-15 15:41:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000022432_22970368.pth... -[2023-10-15 15:41:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000022368_22904832.pth... -[2023-10-15 15:41:43,482][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000020768_21266432.pth -[2023-10-15 15:41:43,492][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000020704_21200896.pth -[2023-10-15 15:41:43,497][52410] Saving new best policy, reward=32.970! -[2023-10-15 15:41:44,432][52866] Updated weights for policy 1, policy_version 22440 (0.0009) -[2023-10-15 15:41:44,803][52866] Updated weights for policy 1, policy_version 22450 (0.0007) -[2023-10-15 15:41:44,827][52833] Updated weights for policy 0, policy_version 22370 (0.0010) -[2023-10-15 15:41:45,177][52866] Updated weights for policy 1, policy_version 22460 (0.0008) -[2023-10-15 15:41:45,192][52833] Updated weights for policy 0, policy_version 22380 (0.0007) -[2023-10-15 15:41:45,554][52833] Updated weights for policy 0, policy_version 22390 (0.0007) -[2023-10-15 15:41:45,920][52833] Updated weights for policy 0, policy_version 22400 (0.0008) -[2023-10-15 15:41:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 45940736. Throughput: 0: 1783.0, 1: 1791.6. Samples: 11490246. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 15:41:48,441][51532] Avg episode reward: [(0, '32.550'), (1, '30.350')] -[2023-10-15 15:41:49,079][52866] Updated weights for policy 1, policy_version 22470 (0.0008) -[2023-10-15 15:41:49,450][52866] Updated weights for policy 1, policy_version 22480 (0.0007) -[2023-10-15 15:41:49,817][52866] Updated weights for policy 1, policy_version 22490 (0.0009) -[2023-10-15 15:41:49,918][52833] Updated weights for policy 0, policy_version 22410 (0.0007) -[2023-10-15 15:41:50,283][52833] Updated weights for policy 0, policy_version 22420 (0.0008) -[2023-10-15 15:41:50,652][52833] Updated weights for policy 0, policy_version 22430 (0.0008) -[2023-10-15 15:41:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 46006272. Throughput: 0: 1782.1, 1: 1791.4. Samples: 11512648. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:41:53,441][51532] Avg episode reward: [(0, '31.670'), (1, '30.110')] -[2023-10-15 15:41:53,570][52866] Updated weights for policy 1, policy_version 22500 (0.0007) -[2023-10-15 15:41:53,930][52866] Updated weights for policy 1, policy_version 22510 (0.0008) -[2023-10-15 15:41:54,240][52833] Updated weights for policy 0, policy_version 22440 (0.0008) -[2023-10-15 15:41:54,293][52866] Updated weights for policy 1, policy_version 22520 (0.0007) -[2023-10-15 15:41:54,608][52833] Updated weights for policy 0, policy_version 22450 (0.0007) -[2023-10-15 15:41:54,979][52833] Updated weights for policy 0, policy_version 22460 (0.0008) -[2023-10-15 15:41:58,207][52866] Updated weights for policy 1, policy_version 22530 (0.0007) -[2023-10-15 15:41:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 46071808. Throughput: 0: 1790.9, 1: 1811.7. Samples: 11535212. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:41:58,441][51532] Avg episode reward: [(0, '31.640'), (1, '30.960')] -[2023-10-15 15:41:58,605][52866] Updated weights for policy 1, policy_version 22540 (0.0008) -[2023-10-15 15:41:58,750][52833] Updated weights for policy 0, policy_version 22470 (0.0007) -[2023-10-15 15:41:58,975][52866] Updated weights for policy 1, policy_version 22550 (0.0008) -[2023-10-15 15:41:59,137][52833] Updated weights for policy 0, policy_version 22480 (0.0007) -[2023-10-15 15:41:59,336][52866] Updated weights for policy 1, policy_version 22560 (0.0007) -[2023-10-15 15:41:59,507][52833] Updated weights for policy 0, policy_version 22490 (0.0007) -[2023-10-15 15:42:02,935][52866] Updated weights for policy 1, policy_version 22570 (0.0008) -[2023-10-15 15:42:03,252][52833] Updated weights for policy 0, policy_version 22500 (0.0009) -[2023-10-15 15:42:03,301][52866] Updated weights for policy 1, policy_version 22580 (0.0009) -[2023-10-15 15:42:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 46137344. Throughput: 0: 1784.7, 1: 1792.5. Samples: 11544838. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:42:03,441][51532] Avg episode reward: [(0, '31.450'), (1, '31.030')] -[2023-10-15 15:42:03,628][52833] Updated weights for policy 0, policy_version 22510 (0.0007) -[2023-10-15 15:42:03,667][52866] Updated weights for policy 1, policy_version 22590 (0.0008) -[2023-10-15 15:42:03,992][52833] Updated weights for policy 0, policy_version 22520 (0.0009) -[2023-10-15 15:42:07,306][52866] Updated weights for policy 1, policy_version 22600 (0.0010) -[2023-10-15 15:42:07,675][52866] Updated weights for policy 1, policy_version 22610 (0.0010) -[2023-10-15 15:42:07,792][52833] Updated weights for policy 0, policy_version 22530 (0.0010) -[2023-10-15 15:42:08,037][52866] Updated weights for policy 1, policy_version 22620 (0.0007) -[2023-10-15 15:42:08,156][52833] Updated weights for policy 0, policy_version 22540 (0.0007) -[2023-10-15 15:42:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 46235648. Throughput: 0: 1780.0, 1: 1815.4. Samples: 11567090. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 15:42:08,441][51532] Avg episode reward: [(0, '31.200'), (1, '30.090')] -[2023-10-15 15:42:08,532][52833] Updated weights for policy 0, policy_version 22550 (0.0007) -[2023-10-15 15:42:08,902][52833] Updated weights for policy 0, policy_version 22560 (0.0010) -[2023-10-15 15:42:11,864][52866] Updated weights for policy 1, policy_version 22630 (0.0008) -[2023-10-15 15:42:12,234][52866] Updated weights for policy 1, policy_version 22640 (0.0009) -[2023-10-15 15:42:12,612][52866] Updated weights for policy 1, policy_version 22650 (0.0009) -[2023-10-15 15:42:12,790][52833] Updated weights for policy 0, policy_version 22570 (0.0007) -[2023-10-15 15:42:13,162][52833] Updated weights for policy 0, policy_version 22580 (0.0011) -[2023-10-15 15:42:13,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 46301184. Throughput: 0: 1796.8, 1: 1794.7. Samples: 11587434. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:42:13,442][51532] Avg episode reward: [(0, '30.620'), (1, '30.730')] -[2023-10-15 15:42:13,527][52833] Updated weights for policy 0, policy_version 22590 (0.0010) -[2023-10-15 15:42:16,270][52866] Updated weights for policy 1, policy_version 22660 (0.0007) -[2023-10-15 15:42:16,631][52866] Updated weights for policy 1, policy_version 22670 (0.0007) -[2023-10-15 15:42:17,000][52866] Updated weights for policy 1, policy_version 22680 (0.0009) -[2023-10-15 15:42:17,337][52833] Updated weights for policy 0, policy_version 22600 (0.0008) -[2023-10-15 15:42:17,713][52833] Updated weights for policy 0, policy_version 22610 (0.0011) -[2023-10-15 15:42:18,089][52833] Updated weights for policy 0, policy_version 22620 (0.0010) -[2023-10-15 15:42:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 46399488. Throughput: 0: 1773.2, 1: 1816.9. Samples: 11599336. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:42:18,441][51532] Avg episode reward: [(0, '32.430'), (1, '30.140')] -[2023-10-15 15:42:20,845][52866] Updated weights for policy 1, policy_version 22690 (0.0009) -[2023-10-15 15:42:21,208][52866] Updated weights for policy 1, policy_version 22700 (0.0008) -[2023-10-15 15:42:21,581][52866] Updated weights for policy 1, policy_version 22710 (0.0008) -[2023-10-15 15:42:21,839][52833] Updated weights for policy 0, policy_version 22630 (0.0009) -[2023-10-15 15:42:21,941][52866] Updated weights for policy 1, policy_version 22720 (0.0007) -[2023-10-15 15:42:22,211][52833] Updated weights for policy 0, policy_version 22640 (0.0008) -[2023-10-15 15:42:22,583][52833] Updated weights for policy 0, policy_version 22650 (0.0008) -[2023-10-15 15:42:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 46465024. Throughput: 0: 1799.7, 1: 1797.1. Samples: 11619832. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:42:23,442][51532] Avg episode reward: [(0, '32.650'), (1, '29.290')] -[2023-10-15 15:42:25,520][52866] Updated weights for policy 1, policy_version 22730 (0.0008) -[2023-10-15 15:42:25,893][52866] Updated weights for policy 1, policy_version 22740 (0.0009) -[2023-10-15 15:42:26,255][52866] Updated weights for policy 1, policy_version 22750 (0.0008) -[2023-10-15 15:42:26,339][52833] Updated weights for policy 0, policy_version 22660 (0.0008) -[2023-10-15 15:42:26,712][52833] Updated weights for policy 0, policy_version 22670 (0.0010) -[2023-10-15 15:42:27,081][52833] Updated weights for policy 0, policy_version 22680 (0.0008) -[2023-10-15 15:42:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 46530560. Throughput: 0: 1775.1, 1: 1801.3. Samples: 11641188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:42:28,442][51532] Avg episode reward: [(0, '34.020'), (1, '30.320')] -[2023-10-15 15:42:28,457][52410] Saving new best policy, reward=34.020! -[2023-10-15 15:42:29,903][52866] Updated weights for policy 1, policy_version 22760 (0.0008) -[2023-10-15 15:42:30,273][52866] Updated weights for policy 1, policy_version 22770 (0.0008) -[2023-10-15 15:42:30,650][52866] Updated weights for policy 1, policy_version 22780 (0.0008) -[2023-10-15 15:42:31,003][52833] Updated weights for policy 0, policy_version 22690 (0.0008) -[2023-10-15 15:42:31,374][52833] Updated weights for policy 0, policy_version 22700 (0.0008) -[2023-10-15 15:42:31,751][52833] Updated weights for policy 0, policy_version 22710 (0.0007) -[2023-10-15 15:42:32,129][52833] Updated weights for policy 0, policy_version 22720 (0.0007) -[2023-10-15 15:42:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 46596096. Throughput: 0: 1799.7, 1: 1799.0. Samples: 11652186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:42:33,442][51532] Avg episode reward: [(0, '31.010'), (1, '31.150')] -[2023-10-15 15:42:34,460][52866] Updated weights for policy 1, policy_version 22790 (0.0011) -[2023-10-15 15:42:34,824][52866] Updated weights for policy 1, policy_version 22800 (0.0010) -[2023-10-15 15:42:35,194][52866] Updated weights for policy 1, policy_version 22810 (0.0010) -[2023-10-15 15:42:36,000][52833] Updated weights for policy 0, policy_version 22730 (0.0009) -[2023-10-15 15:42:36,374][52833] Updated weights for policy 0, policy_version 22740 (0.0009) -[2023-10-15 15:42:36,744][52833] Updated weights for policy 0, policy_version 22750 (0.0007) -[2023-10-15 15:42:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 46661632. Throughput: 0: 1767.1, 1: 1798.8. Samples: 11673112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:42:38,442][51532] Avg episode reward: [(0, '30.430'), (1, '31.740')] -[2023-10-15 15:42:38,908][52866] Updated weights for policy 1, policy_version 22820 (0.0009) -[2023-10-15 15:42:39,284][52866] Updated weights for policy 1, policy_version 22830 (0.0007) -[2023-10-15 15:42:39,642][52866] Updated weights for policy 1, policy_version 22840 (0.0010) -[2023-10-15 15:42:40,502][52833] Updated weights for policy 0, policy_version 22760 (0.0008) -[2023-10-15 15:42:40,869][52833] Updated weights for policy 0, policy_version 22770 (0.0010) -[2023-10-15 15:42:41,247][52833] Updated weights for policy 0, policy_version 22780 (0.0011) -[2023-10-15 15:42:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 46727168. Throughput: 0: 1760.9, 1: 1804.5. Samples: 11695656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:42:43,442][51532] Avg episode reward: [(0, '31.850'), (1, '34.860')] -[2023-10-15 15:42:43,455][52866] Updated weights for policy 1, policy_version 22850 (0.0009) -[2023-10-15 15:42:43,852][52866] Updated weights for policy 1, policy_version 22860 (0.0010) -[2023-10-15 15:42:44,223][52866] Updated weights for policy 1, policy_version 22870 (0.0011) -[2023-10-15 15:42:44,584][52518] Saving new best policy, reward=34.860! -[2023-10-15 15:42:44,585][52866] Updated weights for policy 1, policy_version 22880 (0.0008) -[2023-10-15 15:42:45,005][52833] Updated weights for policy 0, policy_version 22790 (0.0007) -[2023-10-15 15:42:45,379][52833] Updated weights for policy 0, policy_version 22800 (0.0008) -[2023-10-15 15:42:45,753][52833] Updated weights for policy 0, policy_version 22810 (0.0007) -[2023-10-15 15:42:48,297][52866] Updated weights for policy 1, policy_version 22890 (0.0007) -[2023-10-15 15:42:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 46792704. Throughput: 0: 1772.2, 1: 1806.7. Samples: 11705886. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-15 15:42:48,442][51532] Avg episode reward: [(0, '32.610'), (1, '33.850')] -[2023-10-15 15:42:48,657][52866] Updated weights for policy 1, policy_version 22900 (0.0009) -[2023-10-15 15:42:49,023][52866] Updated weights for policy 1, policy_version 22910 (0.0009) -[2023-10-15 15:42:49,551][52833] Updated weights for policy 0, policy_version 22820 (0.0008) -[2023-10-15 15:42:49,918][52833] Updated weights for policy 0, policy_version 22830 (0.0009) -[2023-10-15 15:42:50,286][52833] Updated weights for policy 0, policy_version 22840 (0.0007) -[2023-10-15 15:42:52,721][52866] Updated weights for policy 1, policy_version 22920 (0.0007) -[2023-10-15 15:42:53,088][52866] Updated weights for policy 1, policy_version 22930 (0.0008) -[2023-10-15 15:42:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 46858240. Throughput: 0: 1771.5, 1: 1801.5. Samples: 11727872. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-15 15:42:53,442][51532] Avg episode reward: [(0, '31.720'), (1, '32.820')] -[2023-10-15 15:42:53,455][52866] Updated weights for policy 1, policy_version 22940 (0.0009) -[2023-10-15 15:42:53,848][52833] Updated weights for policy 0, policy_version 22850 (0.0009) -[2023-10-15 15:42:54,231][52833] Updated weights for policy 0, policy_version 22860 (0.0008) -[2023-10-15 15:42:54,604][52833] Updated weights for policy 0, policy_version 22870 (0.0008) -[2023-10-15 15:42:54,963][52833] Updated weights for policy 0, policy_version 22880 (0.0009) -[2023-10-15 15:42:57,087][52866] Updated weights for policy 1, policy_version 22950 (0.0007) -[2023-10-15 15:42:57,458][52866] Updated weights for policy 1, policy_version 22960 (0.0008) -[2023-10-15 15:42:57,827][52866] Updated weights for policy 1, policy_version 22970 (0.0009) -[2023-10-15 15:42:58,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 46956544. Throughput: 0: 1794.2, 1: 1810.0. Samples: 11749620. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-15 15:42:58,442][51532] Avg episode reward: [(0, '29.440'), (1, '33.720')] -[2023-10-15 15:42:58,535][52833] Updated weights for policy 0, policy_version 22890 (0.0008) -[2023-10-15 15:42:58,911][52833] Updated weights for policy 0, policy_version 22900 (0.0008) -[2023-10-15 15:42:59,290][52833] Updated weights for policy 0, policy_version 22910 (0.0010) -[2023-10-15 15:43:01,646][52866] Updated weights for policy 1, policy_version 22980 (0.0008) -[2023-10-15 15:43:02,017][52866] Updated weights for policy 1, policy_version 22990 (0.0008) -[2023-10-15 15:43:02,379][52866] Updated weights for policy 1, policy_version 23000 (0.0007) -[2023-10-15 15:43:02,967][52833] Updated weights for policy 0, policy_version 22920 (0.0007) -[2023-10-15 15:43:03,340][52833] Updated weights for policy 0, policy_version 22930 (0.0007) -[2023-10-15 15:43:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 47022080. Throughput: 0: 1785.2, 1: 1800.7. Samples: 11760700. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-15 15:43:03,441][51532] Avg episode reward: [(0, '29.750'), (1, '33.880')] -[2023-10-15 15:43:03,715][52833] Updated weights for policy 0, policy_version 22940 (0.0007) -[2023-10-15 15:43:06,072][52866] Updated weights for policy 1, policy_version 23010 (0.0009) -[2023-10-15 15:43:06,433][52866] Updated weights for policy 1, policy_version 23020 (0.0008) -[2023-10-15 15:43:06,801][52866] Updated weights for policy 1, policy_version 23030 (0.0009) -[2023-10-15 15:43:07,176][52866] Updated weights for policy 1, policy_version 23040 (0.0008) -[2023-10-15 15:43:07,531][52833] Updated weights for policy 0, policy_version 22950 (0.0007) -[2023-10-15 15:43:07,901][52833] Updated weights for policy 0, policy_version 22960 (0.0010) -[2023-10-15 15:43:08,274][52833] Updated weights for policy 0, policy_version 22970 (0.0009) -[2023-10-15 15:43:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 47087616. Throughput: 0: 1794.0, 1: 1811.8. Samples: 11782094. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) -[2023-10-15 15:43:08,442][51532] Avg episode reward: [(0, '29.750'), (1, '34.920')] -[2023-10-15 15:43:08,443][52518] Saving new best policy, reward=34.920! -[2023-10-15 15:43:10,894][52866] Updated weights for policy 1, policy_version 23050 (0.0007) -[2023-10-15 15:43:11,255][52866] Updated weights for policy 1, policy_version 23060 (0.0008) -[2023-10-15 15:43:11,621][52866] Updated weights for policy 1, policy_version 23070 (0.0007) -[2023-10-15 15:43:11,891][52833] Updated weights for policy 0, policy_version 22980 (0.0010) -[2023-10-15 15:43:12,266][52833] Updated weights for policy 0, policy_version 22990 (0.0009) -[2023-10-15 15:43:12,632][52833] Updated weights for policy 0, policy_version 23000 (0.0008) -[2023-10-15 15:43:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 47185920. Throughput: 0: 1793.6, 1: 1809.8. Samples: 11803342. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) -[2023-10-15 15:43:13,441][51532] Avg episode reward: [(0, '28.850'), (1, '34.130')] -[2023-10-15 15:43:15,483][52866] Updated weights for policy 1, policy_version 23080 (0.0008) -[2023-10-15 15:43:15,845][52866] Updated weights for policy 1, policy_version 23090 (0.0007) -[2023-10-15 15:43:16,214][52866] Updated weights for policy 1, policy_version 23100 (0.0007) -[2023-10-15 15:43:16,409][52833] Updated weights for policy 0, policy_version 23010 (0.0009) -[2023-10-15 15:43:16,770][52833] Updated weights for policy 0, policy_version 23020 (0.0010) -[2023-10-15 15:43:17,143][52833] Updated weights for policy 0, policy_version 23030 (0.0007) -[2023-10-15 15:43:17,523][52833] Updated weights for policy 0, policy_version 23040 (0.0008) -[2023-10-15 15:43:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 47251456. Throughput: 0: 1796.7, 1: 1820.6. Samples: 11814966. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) -[2023-10-15 15:43:18,442][51532] Avg episode reward: [(0, '29.930'), (1, '33.170')] -[2023-10-15 15:43:19,929][52866] Updated weights for policy 1, policy_version 23110 (0.0010) -[2023-10-15 15:43:20,305][52866] Updated weights for policy 1, policy_version 23120 (0.0011) -[2023-10-15 15:43:20,669][52866] Updated weights for policy 1, policy_version 23130 (0.0011) -[2023-10-15 15:43:21,392][52833] Updated weights for policy 0, policy_version 23050 (0.0007) -[2023-10-15 15:43:21,761][52833] Updated weights for policy 0, policy_version 23060 (0.0008) -[2023-10-15 15:43:22,129][52833] Updated weights for policy 0, policy_version 23070 (0.0010) -[2023-10-15 15:43:23,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 47316992. Throughput: 0: 1806.5, 1: 1807.6. Samples: 11835750. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 15:43:23,442][51532] Avg episode reward: [(0, '27.770'), (1, '33.980')] -[2023-10-15 15:43:24,427][52866] Updated weights for policy 1, policy_version 23140 (0.0009) -[2023-10-15 15:43:24,799][52866] Updated weights for policy 1, policy_version 23150 (0.0008) -[2023-10-15 15:43:25,177][52866] Updated weights for policy 1, policy_version 23160 (0.0009) -[2023-10-15 15:43:25,610][52833] Updated weights for policy 0, policy_version 23080 (0.0008) -[2023-10-15 15:43:25,984][52833] Updated weights for policy 0, policy_version 23090 (0.0010) -[2023-10-15 15:43:26,362][52833] Updated weights for policy 0, policy_version 23100 (0.0010) -[2023-10-15 15:43:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 47382528. Throughput: 0: 1800.2, 1: 1802.7. Samples: 11857786. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 15:43:28,442][51532] Avg episode reward: [(0, '28.310'), (1, '31.550')] -[2023-10-15 15:43:28,947][52866] Updated weights for policy 1, policy_version 23170 (0.0007) -[2023-10-15 15:43:29,347][52866] Updated weights for policy 1, policy_version 23180 (0.0007) -[2023-10-15 15:43:29,706][52866] Updated weights for policy 1, policy_version 23190 (0.0010) -[2023-10-15 15:43:30,071][52866] Updated weights for policy 1, policy_version 23200 (0.0008) -[2023-10-15 15:43:30,333][52833] Updated weights for policy 0, policy_version 23110 (0.0008) -[2023-10-15 15:43:30,719][52833] Updated weights for policy 0, policy_version 23120 (0.0007) -[2023-10-15 15:43:31,094][52833] Updated weights for policy 0, policy_version 23130 (0.0008) -[2023-10-15 15:43:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 47448064. Throughput: 0: 1801.7, 1: 1798.4. Samples: 11867890. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 15:43:33,442][51532] Avg episode reward: [(0, '31.440'), (1, '30.980')] -[2023-10-15 15:43:33,879][52866] Updated weights for policy 1, policy_version 23210 (0.0008) -[2023-10-15 15:43:34,247][52866] Updated weights for policy 1, policy_version 23220 (0.0007) -[2023-10-15 15:43:34,613][52866] Updated weights for policy 1, policy_version 23230 (0.0007) -[2023-10-15 15:43:34,750][52833] Updated weights for policy 0, policy_version 23140 (0.0010) -[2023-10-15 15:43:35,126][52833] Updated weights for policy 0, policy_version 23150 (0.0009) -[2023-10-15 15:43:35,500][52833] Updated weights for policy 0, policy_version 23160 (0.0010) -[2023-10-15 15:43:38,343][52866] Updated weights for policy 1, policy_version 23240 (0.0008) -[2023-10-15 15:43:38,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 47513600. Throughput: 0: 1797.0, 1: 1798.2. Samples: 11889656. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 15:43:38,441][51532] Avg episode reward: [(0, '31.150'), (1, '33.770')] -[2023-10-15 15:43:38,706][52866] Updated weights for policy 1, policy_version 23250 (0.0009) -[2023-10-15 15:43:39,077][52866] Updated weights for policy 1, policy_version 23260 (0.0009) -[2023-10-15 15:43:39,443][52833] Updated weights for policy 0, policy_version 23170 (0.0009) -[2023-10-15 15:43:39,802][52833] Updated weights for policy 0, policy_version 23180 (0.0007) -[2023-10-15 15:43:40,175][52833] Updated weights for policy 0, policy_version 23190 (0.0009) -[2023-10-15 15:43:40,542][52833] Updated weights for policy 0, policy_version 23200 (0.0009) -[2023-10-15 15:43:42,708][52866] Updated weights for policy 1, policy_version 23270 (0.0007) -[2023-10-15 15:43:43,072][52866] Updated weights for policy 1, policy_version 23280 (0.0009) -[2023-10-15 15:43:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 47579136. Throughput: 0: 1790.7, 1: 1813.8. Samples: 11911822. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) -[2023-10-15 15:43:43,441][51532] Avg episode reward: [(0, '30.110'), (1, '34.020')] -[2023-10-15 15:43:43,444][52866] Updated weights for policy 1, policy_version 23290 (0.0009) -[2023-10-15 15:43:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000023200_23756800.pth... -[2023-10-15 15:43:43,489][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000021536_22052864.pth -[2023-10-15 15:43:43,657][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000023296_23855104.pth... -[2023-10-15 15:43:43,686][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000021600_22118400.pth -[2023-10-15 15:43:44,188][52833] Updated weights for policy 0, policy_version 23210 (0.0007) -[2023-10-15 15:43:44,557][52833] Updated weights for policy 0, policy_version 23220 (0.0008) -[2023-10-15 15:43:44,916][52833] Updated weights for policy 0, policy_version 23230 (0.0011) -[2023-10-15 15:43:47,039][52866] Updated weights for policy 1, policy_version 23300 (0.0009) -[2023-10-15 15:43:47,409][52866] Updated weights for policy 1, policy_version 23310 (0.0007) -[2023-10-15 15:43:47,777][52866] Updated weights for policy 1, policy_version 23320 (0.0008) -[2023-10-15 15:43:48,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 47677440. Throughput: 0: 1787.9, 1: 1801.2. Samples: 11922208. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) -[2023-10-15 15:43:48,442][51532] Avg episode reward: [(0, '29.350'), (1, '33.100')] -[2023-10-15 15:43:48,602][52833] Updated weights for policy 0, policy_version 23240 (0.0010) -[2023-10-15 15:43:48,976][52833] Updated weights for policy 0, policy_version 23250 (0.0008) -[2023-10-15 15:43:49,346][52833] Updated weights for policy 0, policy_version 23260 (0.0011) -[2023-10-15 15:43:51,590][52866] Updated weights for policy 1, policy_version 23330 (0.0009) -[2023-10-15 15:43:51,961][52866] Updated weights for policy 1, policy_version 23340 (0.0009) -[2023-10-15 15:43:52,327][52866] Updated weights for policy 1, policy_version 23350 (0.0009) -[2023-10-15 15:43:52,705][52866] Updated weights for policy 1, policy_version 23360 (0.0010) -[2023-10-15 15:43:53,147][52833] Updated weights for policy 0, policy_version 23270 (0.0009) -[2023-10-15 15:43:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 47742976. Throughput: 0: 1784.1, 1: 1812.1. Samples: 11943922. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) -[2023-10-15 15:43:53,442][51532] Avg episode reward: [(0, '31.710'), (1, '33.490')] -[2023-10-15 15:43:53,516][52833] Updated weights for policy 0, policy_version 23280 (0.0011) -[2023-10-15 15:43:53,894][52833] Updated weights for policy 0, policy_version 23290 (0.0007) -[2023-10-15 15:43:56,560][52866] Updated weights for policy 1, policy_version 23370 (0.0008) -[2023-10-15 15:43:56,926][52866] Updated weights for policy 1, policy_version 23380 (0.0009) -[2023-10-15 15:43:57,297][52866] Updated weights for policy 1, policy_version 23390 (0.0008) -[2023-10-15 15:43:57,756][52833] Updated weights for policy 0, policy_version 23300 (0.0008) -[2023-10-15 15:43:58,136][52833] Updated weights for policy 0, policy_version 23310 (0.0007) -[2023-10-15 15:43:58,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 47808512. Throughput: 0: 1804.6, 1: 1786.0. Samples: 11964920. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) -[2023-10-15 15:43:58,441][51532] Avg episode reward: [(0, '32.780'), (1, '33.810')] -[2023-10-15 15:43:58,506][52833] Updated weights for policy 0, policy_version 23320 (0.0007) -[2023-10-15 15:44:00,945][52866] Updated weights for policy 1, policy_version 23400 (0.0008) -[2023-10-15 15:44:01,306][52866] Updated weights for policy 1, policy_version 23410 (0.0010) -[2023-10-15 15:44:01,673][52866] Updated weights for policy 1, policy_version 23420 (0.0008) -[2023-10-15 15:44:02,227][52833] Updated weights for policy 0, policy_version 23330 (0.0007) -[2023-10-15 15:44:02,611][52833] Updated weights for policy 0, policy_version 23340 (0.0008) -[2023-10-15 15:44:02,968][52833] Updated weights for policy 0, policy_version 23350 (0.0009) -[2023-10-15 15:44:03,341][52833] Updated weights for policy 0, policy_version 23360 (0.0007) -[2023-10-15 15:44:03,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 47906816. Throughput: 0: 1784.2, 1: 1802.6. Samples: 11976372. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-15 15:44:03,441][51532] Avg episode reward: [(0, '32.280'), (1, '34.890')] -[2023-10-15 15:44:05,283][52866] Updated weights for policy 1, policy_version 23430 (0.0008) -[2023-10-15 15:44:05,661][52866] Updated weights for policy 1, policy_version 23440 (0.0009) -[2023-10-15 15:44:06,041][52866] Updated weights for policy 1, policy_version 23450 (0.0009) -[2023-10-15 15:44:07,025][52833] Updated weights for policy 0, policy_version 23370 (0.0009) -[2023-10-15 15:44:07,394][52833] Updated weights for policy 0, policy_version 23380 (0.0009) -[2023-10-15 15:44:07,758][52833] Updated weights for policy 0, policy_version 23390 (0.0010) -[2023-10-15 15:44:08,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 47972352. Throughput: 0: 1805.9, 1: 1791.0. Samples: 11997610. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-15 15:44:08,442][51532] Avg episode reward: [(0, '27.610'), (1, '35.430')] -[2023-10-15 15:44:08,443][52518] Saving new best policy, reward=35.430! -[2023-10-15 15:44:09,694][52866] Updated weights for policy 1, policy_version 23460 (0.0008) -[2023-10-15 15:44:10,061][52866] Updated weights for policy 1, policy_version 23470 (0.0010) -[2023-10-15 15:44:10,444][52866] Updated weights for policy 1, policy_version 23480 (0.0009) -[2023-10-15 15:44:11,511][52833] Updated weights for policy 0, policy_version 23400 (0.0009) -[2023-10-15 15:44:11,876][52833] Updated weights for policy 0, policy_version 23410 (0.0007) -[2023-10-15 15:44:12,246][52833] Updated weights for policy 0, policy_version 23420 (0.0008) -[2023-10-15 15:44:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 48037888. Throughput: 0: 1790.0, 1: 1793.2. Samples: 12019028. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-15 15:44:13,441][51532] Avg episode reward: [(0, '29.190'), (1, '37.760')] -[2023-10-15 15:44:13,452][52518] Saving new best policy, reward=37.760! -[2023-10-15 15:44:14,263][52866] Updated weights for policy 1, policy_version 23490 (0.0009) -[2023-10-15 15:44:14,665][52866] Updated weights for policy 1, policy_version 23500 (0.0007) -[2023-10-15 15:44:15,034][52866] Updated weights for policy 1, policy_version 23510 (0.0007) -[2023-10-15 15:44:15,388][52866] Updated weights for policy 1, policy_version 23520 (0.0007) -[2023-10-15 15:44:16,036][52833] Updated weights for policy 0, policy_version 23430 (0.0008) -[2023-10-15 15:44:16,435][52833] Updated weights for policy 0, policy_version 23440 (0.0009) -[2023-10-15 15:44:16,810][52833] Updated weights for policy 0, policy_version 23450 (0.0008) -[2023-10-15 15:44:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 48103424. Throughput: 0: 1814.4, 1: 1794.5. Samples: 12030294. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:44:18,442][51532] Avg episode reward: [(0, '28.020'), (1, '35.420')] -[2023-10-15 15:44:19,115][52866] Updated weights for policy 1, policy_version 23530 (0.0007) -[2023-10-15 15:44:19,486][52866] Updated weights for policy 1, policy_version 23540 (0.0011) -[2023-10-15 15:44:19,859][52866] Updated weights for policy 1, policy_version 23550 (0.0010) -[2023-10-15 15:44:20,476][52833] Updated weights for policy 0, policy_version 23460 (0.0008) -[2023-10-15 15:44:20,850][52833] Updated weights for policy 0, policy_version 23470 (0.0007) -[2023-10-15 15:44:21,211][52833] Updated weights for policy 0, policy_version 23480 (0.0009) -[2023-10-15 15:44:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 48168960. Throughput: 0: 1799.2, 1: 1795.3. Samples: 12051408. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:44:23,441][51532] Avg episode reward: [(0, '30.330'), (1, '31.870')] -[2023-10-15 15:44:23,487][52866] Updated weights for policy 1, policy_version 23560 (0.0008) -[2023-10-15 15:44:23,857][52866] Updated weights for policy 1, policy_version 23570 (0.0008) -[2023-10-15 15:44:24,229][52866] Updated weights for policy 1, policy_version 23580 (0.0010) -[2023-10-15 15:44:24,905][52833] Updated weights for policy 0, policy_version 23490 (0.0010) -[2023-10-15 15:44:25,278][52833] Updated weights for policy 0, policy_version 23500 (0.0007) -[2023-10-15 15:44:25,651][52833] Updated weights for policy 0, policy_version 23510 (0.0009) -[2023-10-15 15:44:26,021][52833] Updated weights for policy 0, policy_version 23520 (0.0008) -[2023-10-15 15:44:28,120][52866] Updated weights for policy 1, policy_version 23590 (0.0007) -[2023-10-15 15:44:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 48234496. Throughput: 0: 1797.7, 1: 1807.4. Samples: 12074050. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:44:28,441][51532] Avg episode reward: [(0, '27.740'), (1, '31.500')] -[2023-10-15 15:44:28,493][52866] Updated weights for policy 1, policy_version 23600 (0.0008) -[2023-10-15 15:44:28,854][52866] Updated weights for policy 1, policy_version 23610 (0.0008) -[2023-10-15 15:44:29,779][52833] Updated weights for policy 0, policy_version 23530 (0.0009) -[2023-10-15 15:44:30,145][52833] Updated weights for policy 0, policy_version 23540 (0.0007) -[2023-10-15 15:44:30,510][52833] Updated weights for policy 0, policy_version 23550 (0.0007) -[2023-10-15 15:44:32,650][52866] Updated weights for policy 1, policy_version 23620 (0.0008) -[2023-10-15 15:44:33,019][52866] Updated weights for policy 1, policy_version 23630 (0.0007) -[2023-10-15 15:44:33,379][52866] Updated weights for policy 1, policy_version 23640 (0.0010) -[2023-10-15 15:44:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 48300032. Throughput: 0: 1802.9, 1: 1795.7. Samples: 12084146. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:44:33,442][51532] Avg episode reward: [(0, '29.510'), (1, '32.860')] -[2023-10-15 15:44:34,228][52833] Updated weights for policy 0, policy_version 23560 (0.0009) -[2023-10-15 15:44:34,605][52833] Updated weights for policy 0, policy_version 23570 (0.0010) -[2023-10-15 15:44:34,979][52833] Updated weights for policy 0, policy_version 23580 (0.0008) -[2023-10-15 15:44:37,066][52866] Updated weights for policy 1, policy_version 23650 (0.0009) -[2023-10-15 15:44:37,423][52866] Updated weights for policy 1, policy_version 23660 (0.0010) -[2023-10-15 15:44:37,794][52866] Updated weights for policy 1, policy_version 23670 (0.0010) -[2023-10-15 15:44:38,162][52866] Updated weights for policy 1, policy_version 23680 (0.0008) -[2023-10-15 15:44:38,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 48398336. Throughput: 0: 1803.2, 1: 1810.0. Samples: 12106512. Policy #0 lag: (min: 26.0, avg: 29.2, max: 51.0) -[2023-10-15 15:44:38,442][51532] Avg episode reward: [(0, '29.630'), (1, '31.970')] -[2023-10-15 15:44:38,713][52833] Updated weights for policy 0, policy_version 23590 (0.0007) -[2023-10-15 15:44:39,089][52833] Updated weights for policy 0, policy_version 23600 (0.0007) -[2023-10-15 15:44:39,463][52833] Updated weights for policy 0, policy_version 23610 (0.0008) -[2023-10-15 15:44:41,739][52866] Updated weights for policy 1, policy_version 23690 (0.0007) -[2023-10-15 15:44:42,106][52866] Updated weights for policy 1, policy_version 23700 (0.0007) -[2023-10-15 15:44:42,471][52866] Updated weights for policy 1, policy_version 23710 (0.0008) -[2023-10-15 15:44:43,135][52833] Updated weights for policy 0, policy_version 23620 (0.0009) -[2023-10-15 15:44:43,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 48463872. Throughput: 0: 1817.3, 1: 1803.7. Samples: 12127864. Policy #0 lag: (min: 26.0, avg: 29.2, max: 51.0) -[2023-10-15 15:44:43,442][51532] Avg episode reward: [(0, '30.250'), (1, '31.680')] -[2023-10-15 15:44:43,506][52833] Updated weights for policy 0, policy_version 23630 (0.0009) -[2023-10-15 15:44:43,882][52833] Updated weights for policy 0, policy_version 23640 (0.0009) -[2023-10-15 15:44:46,210][52866] Updated weights for policy 1, policy_version 23720 (0.0008) -[2023-10-15 15:44:46,579][52866] Updated weights for policy 1, policy_version 23730 (0.0007) -[2023-10-15 15:44:46,936][52866] Updated weights for policy 1, policy_version 23740 (0.0007) -[2023-10-15 15:44:47,582][52833] Updated weights for policy 0, policy_version 23650 (0.0010) -[2023-10-15 15:44:47,950][52833] Updated weights for policy 0, policy_version 23660 (0.0009) -[2023-10-15 15:44:48,314][52833] Updated weights for policy 0, policy_version 23670 (0.0010) -[2023-10-15 15:44:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 48529408. Throughput: 0: 1809.1, 1: 1811.1. Samples: 12139278. Policy #0 lag: (min: 26.0, avg: 29.2, max: 51.0) -[2023-10-15 15:44:48,441][51532] Avg episode reward: [(0, '29.630'), (1, '31.990')] -[2023-10-15 15:44:48,688][52833] Updated weights for policy 0, policy_version 23680 (0.0009) -[2023-10-15 15:44:50,619][52866] Updated weights for policy 1, policy_version 23750 (0.0009) -[2023-10-15 15:44:50,988][52866] Updated weights for policy 1, policy_version 23760 (0.0011) -[2023-10-15 15:44:51,355][52866] Updated weights for policy 1, policy_version 23770 (0.0010) -[2023-10-15 15:44:52,387][52833] Updated weights for policy 0, policy_version 23690 (0.0007) -[2023-10-15 15:44:52,757][52833] Updated weights for policy 0, policy_version 23700 (0.0007) -[2023-10-15 15:44:53,127][52833] Updated weights for policy 0, policy_version 23710 (0.0008) -[2023-10-15 15:44:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 48627712. Throughput: 0: 1815.5, 1: 1805.0. Samples: 12160532. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) -[2023-10-15 15:44:53,442][51532] Avg episode reward: [(0, '30.000'), (1, '29.040')] -[2023-10-15 15:44:55,126][52866] Updated weights for policy 1, policy_version 23780 (0.0008) -[2023-10-15 15:44:55,493][52866] Updated weights for policy 1, policy_version 23790 (0.0007) -[2023-10-15 15:44:55,858][52866] Updated weights for policy 1, policy_version 23800 (0.0007) -[2023-10-15 15:44:56,768][52833] Updated weights for policy 0, policy_version 23720 (0.0009) -[2023-10-15 15:44:57,142][52833] Updated weights for policy 0, policy_version 23730 (0.0010) -[2023-10-15 15:44:57,515][52833] Updated weights for policy 0, policy_version 23740 (0.0009) -[2023-10-15 15:44:58,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 48693248. Throughput: 0: 1804.7, 1: 1808.7. Samples: 12181630. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) -[2023-10-15 15:44:58,442][51532] Avg episode reward: [(0, '30.600'), (1, '31.830')] -[2023-10-15 15:44:59,674][52866] Updated weights for policy 1, policy_version 23810 (0.0009) -[2023-10-15 15:45:00,070][52866] Updated weights for policy 1, policy_version 23820 (0.0011) -[2023-10-15 15:45:00,434][52866] Updated weights for policy 1, policy_version 23830 (0.0008) -[2023-10-15 15:45:00,806][52866] Updated weights for policy 1, policy_version 23840 (0.0012) -[2023-10-15 15:45:01,150][52833] Updated weights for policy 0, policy_version 23750 (0.0008) -[2023-10-15 15:45:01,527][52833] Updated weights for policy 0, policy_version 23760 (0.0010) -[2023-10-15 15:45:01,897][52833] Updated weights for policy 0, policy_version 23770 (0.0010) -[2023-10-15 15:45:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 48758784. Throughput: 0: 1809.4, 1: 1802.5. Samples: 12192826. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) -[2023-10-15 15:45:03,442][51532] Avg episode reward: [(0, '29.490'), (1, '33.390')] -[2023-10-15 15:45:04,684][52866] Updated weights for policy 1, policy_version 23850 (0.0011) -[2023-10-15 15:45:05,059][52866] Updated weights for policy 1, policy_version 23860 (0.0008) -[2023-10-15 15:45:05,432][52866] Updated weights for policy 1, policy_version 23870 (0.0009) -[2023-10-15 15:45:05,627][52833] Updated weights for policy 0, policy_version 23780 (0.0008) -[2023-10-15 15:45:06,001][52833] Updated weights for policy 0, policy_version 23790 (0.0007) -[2023-10-15 15:45:06,370][52833] Updated weights for policy 0, policy_version 23800 (0.0010) -[2023-10-15 15:45:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 48824320. Throughput: 0: 1808.5, 1: 1798.4. Samples: 12213718. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) -[2023-10-15 15:45:08,441][51532] Avg episode reward: [(0, '29.650'), (1, '32.820')] -[2023-10-15 15:45:09,325][52866] Updated weights for policy 1, policy_version 23880 (0.0007) -[2023-10-15 15:45:09,687][52866] Updated weights for policy 1, policy_version 23890 (0.0009) -[2023-10-15 15:45:10,056][52866] Updated weights for policy 1, policy_version 23900 (0.0010) -[2023-10-15 15:45:10,182][52833] Updated weights for policy 0, policy_version 23810 (0.0010) -[2023-10-15 15:45:10,547][52833] Updated weights for policy 0, policy_version 23820 (0.0007) -[2023-10-15 15:45:10,911][52833] Updated weights for policy 0, policy_version 23830 (0.0008) -[2023-10-15 15:45:11,280][52833] Updated weights for policy 0, policy_version 23840 (0.0009) -[2023-10-15 15:45:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 48889856. Throughput: 0: 1809.1, 1: 1793.4. Samples: 12236162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:45:13,442][51532] Avg episode reward: [(0, '29.900'), (1, '32.840')] -[2023-10-15 15:45:13,748][52866] Updated weights for policy 1, policy_version 23910 (0.0009) -[2023-10-15 15:45:14,119][52866] Updated weights for policy 1, policy_version 23920 (0.0008) -[2023-10-15 15:45:14,482][52866] Updated weights for policy 1, policy_version 23930 (0.0010) -[2023-10-15 15:45:14,934][52833] Updated weights for policy 0, policy_version 23850 (0.0008) -[2023-10-15 15:45:15,310][52833] Updated weights for policy 0, policy_version 23860 (0.0008) -[2023-10-15 15:45:15,680][52833] Updated weights for policy 0, policy_version 23870 (0.0008) -[2023-10-15 15:45:18,306][52866] Updated weights for policy 1, policy_version 23940 (0.0008) -[2023-10-15 15:45:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 48955392. Throughput: 0: 1805.1, 1: 1790.2. Samples: 12245936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:45:18,442][51532] Avg episode reward: [(0, '32.500'), (1, '32.420')] -[2023-10-15 15:45:18,670][52866] Updated weights for policy 1, policy_version 23950 (0.0007) -[2023-10-15 15:45:19,035][52866] Updated weights for policy 1, policy_version 23960 (0.0008) -[2023-10-15 15:45:19,432][52833] Updated weights for policy 0, policy_version 23880 (0.0008) -[2023-10-15 15:45:19,796][52833] Updated weights for policy 0, policy_version 23890 (0.0008) -[2023-10-15 15:45:20,172][52833] Updated weights for policy 0, policy_version 23900 (0.0008) -[2023-10-15 15:45:22,663][52866] Updated weights for policy 1, policy_version 23970 (0.0008) -[2023-10-15 15:45:23,033][52866] Updated weights for policy 1, policy_version 23980 (0.0007) -[2023-10-15 15:45:23,397][52866] Updated weights for policy 1, policy_version 23990 (0.0007) -[2023-10-15 15:45:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 49020928. Throughput: 0: 1804.1, 1: 1788.3. Samples: 12268170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:45:23,442][51532] Avg episode reward: [(0, '33.110'), (1, '31.920')] -[2023-10-15 15:45:23,768][52866] Updated weights for policy 1, policy_version 24000 (0.0007) -[2023-10-15 15:45:24,012][52833] Updated weights for policy 0, policy_version 23910 (0.0008) -[2023-10-15 15:45:24,373][52833] Updated weights for policy 0, policy_version 23920 (0.0010) -[2023-10-15 15:45:24,745][52833] Updated weights for policy 0, policy_version 23930 (0.0009) -[2023-10-15 15:45:27,525][52866] Updated weights for policy 1, policy_version 24010 (0.0010) -[2023-10-15 15:45:27,891][52866] Updated weights for policy 1, policy_version 24020 (0.0009) -[2023-10-15 15:45:28,261][52866] Updated weights for policy 1, policy_version 24030 (0.0010) -[2023-10-15 15:45:28,441][51532] Fps is (10 sec: 16383.2, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 49119232. Throughput: 0: 1794.6, 1: 1799.6. Samples: 12289604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:45:28,443][51532] Avg episode reward: [(0, '32.260'), (1, '32.650')] -[2023-10-15 15:45:28,668][52833] Updated weights for policy 0, policy_version 23940 (0.0010) -[2023-10-15 15:45:29,039][52833] Updated weights for policy 0, policy_version 23950 (0.0009) -[2023-10-15 15:45:29,407][52833] Updated weights for policy 0, policy_version 23960 (0.0011) -[2023-10-15 15:45:32,490][52866] Updated weights for policy 1, policy_version 24040 (0.0008) -[2023-10-15 15:45:32,860][52866] Updated weights for policy 1, policy_version 24050 (0.0007) -[2023-10-15 15:45:33,222][52866] Updated weights for policy 1, policy_version 24060 (0.0007) -[2023-10-15 15:45:33,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 49184768. Throughput: 0: 1784.4, 1: 1775.6. Samples: 12299476. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) -[2023-10-15 15:45:33,441][51532] Avg episode reward: [(0, '31.230'), (1, '33.440')] -[2023-10-15 15:45:33,534][52833] Updated weights for policy 0, policy_version 23970 (0.0008) -[2023-10-15 15:45:33,894][52833] Updated weights for policy 0, policy_version 23980 (0.0008) -[2023-10-15 15:45:34,266][52833] Updated weights for policy 0, policy_version 23990 (0.0008) -[2023-10-15 15:45:34,630][52833] Updated weights for policy 0, policy_version 24000 (0.0010) -[2023-10-15 15:45:37,144][52866] Updated weights for policy 1, policy_version 24070 (0.0009) -[2023-10-15 15:45:37,506][52866] Updated weights for policy 1, policy_version 24080 (0.0009) -[2023-10-15 15:45:37,868][52866] Updated weights for policy 1, policy_version 24090 (0.0008) -[2023-10-15 15:45:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 49250304. Throughput: 0: 1765.5, 1: 1785.8. Samples: 12320340. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) -[2023-10-15 15:45:38,442][51532] Avg episode reward: [(0, '32.150'), (1, '31.760')] -[2023-10-15 15:45:38,575][52833] Updated weights for policy 0, policy_version 24010 (0.0010) -[2023-10-15 15:45:38,943][52833] Updated weights for policy 0, policy_version 24020 (0.0009) -[2023-10-15 15:45:39,312][52833] Updated weights for policy 0, policy_version 24030 (0.0010) -[2023-10-15 15:45:41,947][52866] Updated weights for policy 1, policy_version 24100 (0.0008) -[2023-10-15 15:45:42,306][52866] Updated weights for policy 1, policy_version 24110 (0.0009) -[2023-10-15 15:45:42,674][52866] Updated weights for policy 1, policy_version 24120 (0.0010) -[2023-10-15 15:45:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 49315840. Throughput: 0: 1775.4, 1: 1736.5. Samples: 12339662. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) -[2023-10-15 15:45:43,441][51532] Avg episode reward: [(0, '33.110'), (1, '32.610')] -[2023-10-15 15:45:43,449][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000024128_24707072.pth... -[2023-10-15 15:45:43,481][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000022432_22970368.pth -[2023-10-15 15:45:43,601][52833] Updated weights for policy 0, policy_version 24040 (0.0010) -[2023-10-15 15:45:43,967][52833] Updated weights for policy 0, policy_version 24050 (0.0009) -[2023-10-15 15:45:44,337][52833] Updated weights for policy 0, policy_version 24060 (0.0010) -[2023-10-15 15:45:44,483][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000024064_24641536.pth... -[2023-10-15 15:45:44,523][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000022368_22904832.pth -[2023-10-15 15:45:46,729][52866] Updated weights for policy 1, policy_version 24130 (0.0008) -[2023-10-15 15:45:47,099][52866] Updated weights for policy 1, policy_version 24140 (0.0010) -[2023-10-15 15:45:47,456][52866] Updated weights for policy 1, policy_version 24150 (0.0011) -[2023-10-15 15:45:47,832][52866] Updated weights for policy 1, policy_version 24160 (0.0010) -[2023-10-15 15:45:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 49381376. Throughput: 0: 1726.5, 1: 1762.3. Samples: 12349822. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) -[2023-10-15 15:45:48,441][51532] Avg episode reward: [(0, '33.700'), (1, '31.890')] -[2023-10-15 15:45:48,517][52833] Updated weights for policy 0, policy_version 24070 (0.0010) -[2023-10-15 15:45:48,892][52833] Updated weights for policy 0, policy_version 24080 (0.0007) -[2023-10-15 15:45:49,254][52833] Updated weights for policy 0, policy_version 24090 (0.0009) -[2023-10-15 15:45:51,956][52866] Updated weights for policy 1, policy_version 24170 (0.0011) -[2023-10-15 15:45:52,319][52866] Updated weights for policy 1, policy_version 24180 (0.0010) -[2023-10-15 15:45:52,685][52866] Updated weights for policy 1, policy_version 24190 (0.0007) -[2023-10-15 15:45:53,193][52833] Updated weights for policy 0, policy_version 24100 (0.0009) -[2023-10-15 15:45:53,441][51532] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 14329.1). Total num frames: 49446912. Throughput: 0: 1730.7, 1: 1737.4. Samples: 12369782. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:45:53,442][51532] Avg episode reward: [(0, '30.990'), (1, '30.930')] -[2023-10-15 15:45:53,554][52833] Updated weights for policy 0, policy_version 24110 (0.0009) -[2023-10-15 15:45:53,931][52833] Updated weights for policy 0, policy_version 24120 (0.0009) -[2023-10-15 15:45:56,555][52866] Updated weights for policy 1, policy_version 24200 (0.0008) -[2023-10-15 15:45:56,928][52866] Updated weights for policy 1, policy_version 24210 (0.0008) -[2023-10-15 15:45:57,288][52866] Updated weights for policy 1, policy_version 24220 (0.0009) -[2023-10-15 15:45:57,908][52833] Updated weights for policy 0, policy_version 24130 (0.0009) -[2023-10-15 15:45:58,274][52833] Updated weights for policy 0, policy_version 24140 (0.0010) -[2023-10-15 15:45:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 14329.1). Total num frames: 49512448. Throughput: 0: 1711.3, 1: 1704.6. Samples: 12389876. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:45:58,442][51532] Avg episode reward: [(0, '30.420'), (1, '32.390')] -[2023-10-15 15:45:58,640][52833] Updated weights for policy 0, policy_version 24150 (0.0010) -[2023-10-15 15:45:59,002][52833] Updated weights for policy 0, policy_version 24160 (0.0010) -[2023-10-15 15:46:01,482][52866] Updated weights for policy 1, policy_version 24230 (0.0009) -[2023-10-15 15:46:01,844][52866] Updated weights for policy 1, policy_version 24240 (0.0009) -[2023-10-15 15:46:02,205][52866] Updated weights for policy 1, policy_version 24250 (0.0011) -[2023-10-15 15:46:03,063][52833] Updated weights for policy 0, policy_version 24170 (0.0010) -[2023-10-15 15:46:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 14329.1). Total num frames: 49577984. Throughput: 0: 1702.3, 1: 1723.6. Samples: 12400100. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:46:03,442][51532] Avg episode reward: [(0, '30.780'), (1, '30.950')] -[2023-10-15 15:46:03,442][52833] Updated weights for policy 0, policy_version 24180 (0.0011) -[2023-10-15 15:46:03,812][52833] Updated weights for policy 0, policy_version 24190 (0.0009) -[2023-10-15 15:46:06,451][52866] Updated weights for policy 1, policy_version 24260 (0.0009) -[2023-10-15 15:46:06,816][52866] Updated weights for policy 1, policy_version 24270 (0.0009) -[2023-10-15 15:46:07,176][52866] Updated weights for policy 1, policy_version 24280 (0.0007) -[2023-10-15 15:46:07,858][52833] Updated weights for policy 0, policy_version 24200 (0.0010) -[2023-10-15 15:46:08,233][52833] Updated weights for policy 0, policy_version 24210 (0.0011) -[2023-10-15 15:46:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 14218.0). Total num frames: 49643520. Throughput: 0: 1689.3, 1: 1686.0. Samples: 12420058. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-15 15:46:08,441][51532] Avg episode reward: [(0, '31.450'), (1, '30.940')] -[2023-10-15 15:46:08,600][52833] Updated weights for policy 0, policy_version 24220 (0.0009) -[2023-10-15 15:46:10,847][52866] Updated weights for policy 1, policy_version 24290 (0.0009) -[2023-10-15 15:46:11,210][52866] Updated weights for policy 1, policy_version 24300 (0.0011) -[2023-10-15 15:46:11,579][52866] Updated weights for policy 1, policy_version 24310 (0.0008) -[2023-10-15 15:46:11,951][52866] Updated weights for policy 1, policy_version 24320 (0.0009) -[2023-10-15 15:46:12,408][52833] Updated weights for policy 0, policy_version 24230 (0.0008) -[2023-10-15 15:46:12,776][52833] Updated weights for policy 0, policy_version 24240 (0.0008) -[2023-10-15 15:46:13,147][52833] Updated weights for policy 0, policy_version 24250 (0.0007) -[2023-10-15 15:46:13,441][51532] Fps is (10 sec: 16383.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 49741824. Throughput: 0: 1673.1, 1: 1697.2. Samples: 12441266. Policy #0 lag: (min: 0.0, avg: 19.4, max: 32.0) -[2023-10-15 15:46:13,442][51532] Avg episode reward: [(0, '33.390'), (1, '30.840')] -[2023-10-15 15:46:15,694][52866] Updated weights for policy 1, policy_version 24330 (0.0008) -[2023-10-15 15:46:16,065][52866] Updated weights for policy 1, policy_version 24340 (0.0008) -[2023-10-15 15:46:16,447][52866] Updated weights for policy 1, policy_version 24350 (0.0007) -[2023-10-15 15:46:16,746][52833] Updated weights for policy 0, policy_version 24260 (0.0008) -[2023-10-15 15:46:17,124][52833] Updated weights for policy 0, policy_version 24270 (0.0011) -[2023-10-15 15:46:17,481][52833] Updated weights for policy 0, policy_version 24280 (0.0011) -[2023-10-15 15:46:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 49807360. Throughput: 0: 1705.8, 1: 1702.3. Samples: 12452840. Policy #0 lag: (min: 0.0, avg: 19.4, max: 32.0) -[2023-10-15 15:46:18,442][51532] Avg episode reward: [(0, '32.430'), (1, '30.470')] -[2023-10-15 15:46:20,147][52866] Updated weights for policy 1, policy_version 24360 (0.0007) -[2023-10-15 15:46:20,510][52866] Updated weights for policy 1, policy_version 24370 (0.0009) -[2023-10-15 15:46:20,876][52866] Updated weights for policy 1, policy_version 24380 (0.0009) -[2023-10-15 15:46:21,377][52833] Updated weights for policy 0, policy_version 24290 (0.0007) -[2023-10-15 15:46:21,751][52833] Updated weights for policy 0, policy_version 24300 (0.0009) -[2023-10-15 15:46:22,130][52833] Updated weights for policy 0, policy_version 24310 (0.0008) -[2023-10-15 15:46:22,495][52833] Updated weights for policy 0, policy_version 24320 (0.0009) -[2023-10-15 15:46:23,441][51532] Fps is (10 sec: 13108.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 49872896. Throughput: 0: 1702.2, 1: 1704.6. Samples: 12473648. Policy #0 lag: (min: 0.0, avg: 19.4, max: 32.0) -[2023-10-15 15:46:23,441][51532] Avg episode reward: [(0, '30.740'), (1, '31.340')] -[2023-10-15 15:46:24,647][52866] Updated weights for policy 1, policy_version 24390 (0.0007) -[2023-10-15 15:46:25,015][52866] Updated weights for policy 1, policy_version 24400 (0.0009) -[2023-10-15 15:46:25,387][52866] Updated weights for policy 1, policy_version 24410 (0.0008) -[2023-10-15 15:46:26,260][52833] Updated weights for policy 0, policy_version 24330 (0.0007) -[2023-10-15 15:46:26,628][52833] Updated weights for policy 0, policy_version 24340 (0.0007) -[2023-10-15 15:46:26,980][52833] Updated weights for policy 0, policy_version 24350 (0.0009) -[2023-10-15 15:46:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 14218.0). Total num frames: 49938432. Throughput: 0: 1709.9, 1: 1750.8. Samples: 12495392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:46:28,442][51532] Avg episode reward: [(0, '31.970'), (1, '32.680')] -[2023-10-15 15:46:29,236][52866] Updated weights for policy 1, policy_version 24420 (0.0008) -[2023-10-15 15:46:29,597][52866] Updated weights for policy 1, policy_version 24430 (0.0009) -[2023-10-15 15:46:29,973][52866] Updated weights for policy 1, policy_version 24440 (0.0008) -[2023-10-15 15:46:30,846][52833] Updated weights for policy 0, policy_version 24360 (0.0010) -[2023-10-15 15:46:31,216][52833] Updated weights for policy 0, policy_version 24370 (0.0008) -[2023-10-15 15:46:31,586][52833] Updated weights for policy 0, policy_version 24380 (0.0009) -[2023-10-15 15:46:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 14218.0). Total num frames: 50003968. Throughput: 0: 1742.9, 1: 1730.8. Samples: 12506140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:46:33,442][51532] Avg episode reward: [(0, '34.080'), (1, '31.490')] -[2023-10-15 15:46:33,443][52410] Saving new best policy, reward=34.080! -[2023-10-15 15:46:33,660][52866] Updated weights for policy 1, policy_version 24450 (0.0007) -[2023-10-15 15:46:34,035][52866] Updated weights for policy 1, policy_version 24460 (0.0008) -[2023-10-15 15:46:34,407][52866] Updated weights for policy 1, policy_version 24470 (0.0010) -[2023-10-15 15:46:34,773][52866] Updated weights for policy 1, policy_version 24480 (0.0010) -[2023-10-15 15:46:35,524][52833] Updated weights for policy 0, policy_version 24390 (0.0009) -[2023-10-15 15:46:35,903][52833] Updated weights for policy 0, policy_version 24400 (0.0008) -[2023-10-15 15:46:36,271][52833] Updated weights for policy 0, policy_version 24410 (0.0009) -[2023-10-15 15:46:38,428][52866] Updated weights for policy 1, policy_version 24490 (0.0008) -[2023-10-15 15:46:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 14218.0). Total num frames: 50069504. Throughput: 0: 1739.8, 1: 1766.0. Samples: 12527546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:46:38,441][51532] Avg episode reward: [(0, '36.430'), (1, '33.970')] -[2023-10-15 15:46:38,442][52410] Saving new best policy, reward=36.430! -[2023-10-15 15:46:38,793][52866] Updated weights for policy 1, policy_version 24500 (0.0007) -[2023-10-15 15:46:39,162][52866] Updated weights for policy 1, policy_version 24510 (0.0009) -[2023-10-15 15:46:39,924][52833] Updated weights for policy 0, policy_version 24420 (0.0008) -[2023-10-15 15:46:40,304][52833] Updated weights for policy 0, policy_version 24430 (0.0008) -[2023-10-15 15:46:40,668][52833] Updated weights for policy 0, policy_version 24440 (0.0012) -[2023-10-15 15:46:42,934][52866] Updated weights for policy 1, policy_version 24520 (0.0009) -[2023-10-15 15:46:43,298][52866] Updated weights for policy 1, policy_version 24530 (0.0010) -[2023-10-15 15:46:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 14218.0). Total num frames: 50135040. Throughput: 0: 1756.5, 1: 1790.9. Samples: 12549510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:46:43,441][51532] Avg episode reward: [(0, '34.260'), (1, '33.280')] -[2023-10-15 15:46:43,674][52866] Updated weights for policy 1, policy_version 24540 (0.0009) -[2023-10-15 15:46:44,391][52833] Updated weights for policy 0, policy_version 24450 (0.0011) -[2023-10-15 15:46:44,757][52833] Updated weights for policy 0, policy_version 24460 (0.0008) -[2023-10-15 15:46:45,124][52833] Updated weights for policy 0, policy_version 24470 (0.0008) -[2023-10-15 15:46:45,500][52833] Updated weights for policy 0, policy_version 24480 (0.0010) -[2023-10-15 15:46:47,401][52866] Updated weights for policy 1, policy_version 24550 (0.0010) -[2023-10-15 15:46:47,771][52866] Updated weights for policy 1, policy_version 24560 (0.0009) -[2023-10-15 15:46:48,139][52866] Updated weights for policy 1, policy_version 24570 (0.0011) -[2023-10-15 15:46:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 50233344. Throughput: 0: 1767.0, 1: 1784.1. Samples: 12559900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:46:48,441][51532] Avg episode reward: [(0, '33.580'), (1, '36.670')] -[2023-10-15 15:46:49,321][52833] Updated weights for policy 0, policy_version 24490 (0.0009) -[2023-10-15 15:46:49,689][52833] Updated weights for policy 0, policy_version 24500 (0.0007) -[2023-10-15 15:46:50,058][52833] Updated weights for policy 0, policy_version 24510 (0.0008) -[2023-10-15 15:46:51,739][52866] Updated weights for policy 1, policy_version 24580 (0.0008) -[2023-10-15 15:46:52,106][52866] Updated weights for policy 1, policy_version 24590 (0.0009) -[2023-10-15 15:46:52,475][52866] Updated weights for policy 1, policy_version 24600 (0.0010) -[2023-10-15 15:46:53,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 50298880. Throughput: 0: 1781.4, 1: 1810.9. Samples: 12581714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:46:53,442][51532] Avg episode reward: [(0, '32.200'), (1, '36.730')] -[2023-10-15 15:46:53,695][52833] Updated weights for policy 0, policy_version 24520 (0.0007) -[2023-10-15 15:46:54,071][52833] Updated weights for policy 0, policy_version 24530 (0.0009) -[2023-10-15 15:46:54,448][52833] Updated weights for policy 0, policy_version 24540 (0.0010) -[2023-10-15 15:46:56,159][52866] Updated weights for policy 1, policy_version 24610 (0.0010) -[2023-10-15 15:46:56,526][52866] Updated weights for policy 1, policy_version 24620 (0.0007) -[2023-10-15 15:46:56,890][52866] Updated weights for policy 1, policy_version 24630 (0.0009) -[2023-10-15 15:46:57,254][52866] Updated weights for policy 1, policy_version 24640 (0.0007) -[2023-10-15 15:46:58,318][52833] Updated weights for policy 0, policy_version 24550 (0.0007) -[2023-10-15 15:46:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 50364416. Throughput: 0: 1804.0, 1: 1799.3. Samples: 12603412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:46:58,442][51532] Avg episode reward: [(0, '34.120'), (1, '38.160')] -[2023-10-15 15:46:58,449][52518] Saving new best policy, reward=38.160! -[2023-10-15 15:46:58,689][52833] Updated weights for policy 0, policy_version 24560 (0.0009) -[2023-10-15 15:46:59,057][52833] Updated weights for policy 0, policy_version 24570 (0.0010) -[2023-10-15 15:47:00,894][52866] Updated weights for policy 1, policy_version 24650 (0.0007) -[2023-10-15 15:47:01,265][52866] Updated weights for policy 1, policy_version 24660 (0.0010) -[2023-10-15 15:47:01,639][52866] Updated weights for policy 1, policy_version 24670 (0.0008) -[2023-10-15 15:47:02,788][52833] Updated weights for policy 0, policy_version 24580 (0.0010) -[2023-10-15 15:47:03,162][52833] Updated weights for policy 0, policy_version 24590 (0.0010) -[2023-10-15 15:47:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 50429952. Throughput: 0: 1780.5, 1: 1804.8. Samples: 12614176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:47:03,441][51532] Avg episode reward: [(0, '34.610'), (1, '35.680')] -[2023-10-15 15:47:03,530][52833] Updated weights for policy 0, policy_version 24600 (0.0007) -[2023-10-15 15:47:05,371][52866] Updated weights for policy 1, policy_version 24680 (0.0009) -[2023-10-15 15:47:05,734][52866] Updated weights for policy 1, policy_version 24690 (0.0009) -[2023-10-15 15:47:06,107][52866] Updated weights for policy 1, policy_version 24700 (0.0007) -[2023-10-15 15:47:07,213][52833] Updated weights for policy 0, policy_version 24610 (0.0008) -[2023-10-15 15:47:07,589][52833] Updated weights for policy 0, policy_version 24620 (0.0008) -[2023-10-15 15:47:07,951][52833] Updated weights for policy 0, policy_version 24630 (0.0009) -[2023-10-15 15:47:08,320][52833] Updated weights for policy 0, policy_version 24640 (0.0008) -[2023-10-15 15:47:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 50528256. Throughput: 0: 1804.0, 1: 1803.8. Samples: 12636000. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-15 15:47:08,442][51532] Avg episode reward: [(0, '34.320'), (1, '36.990')] -[2023-10-15 15:47:09,822][52866] Updated weights for policy 1, policy_version 24710 (0.0007) -[2023-10-15 15:47:10,197][52866] Updated weights for policy 1, policy_version 24720 (0.0007) -[2023-10-15 15:47:10,563][52866] Updated weights for policy 1, policy_version 24730 (0.0008) -[2023-10-15 15:47:11,968][52833] Updated weights for policy 0, policy_version 24650 (0.0009) -[2023-10-15 15:47:12,334][52833] Updated weights for policy 0, policy_version 24660 (0.0009) -[2023-10-15 15:47:12,707][52833] Updated weights for policy 0, policy_version 24670 (0.0010) -[2023-10-15 15:47:13,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 50593792. Throughput: 0: 1786.3, 1: 1800.4. Samples: 12656794. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-15 15:47:13,442][51532] Avg episode reward: [(0, '34.360'), (1, '36.450')] -[2023-10-15 15:47:14,376][52866] Updated weights for policy 1, policy_version 24740 (0.0008) -[2023-10-15 15:47:14,737][52866] Updated weights for policy 1, policy_version 24750 (0.0008) -[2023-10-15 15:47:15,098][52866] Updated weights for policy 1, policy_version 24760 (0.0007) -[2023-10-15 15:47:16,392][52833] Updated weights for policy 0, policy_version 24680 (0.0011) -[2023-10-15 15:47:16,746][52833] Updated weights for policy 0, policy_version 24690 (0.0010) -[2023-10-15 15:47:17,121][52833] Updated weights for policy 0, policy_version 24700 (0.0010) -[2023-10-15 15:47:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 50659328. Throughput: 0: 1799.2, 1: 1798.9. Samples: 12668058. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-15 15:47:18,441][51532] Avg episode reward: [(0, '34.220'), (1, '33.500')] -[2023-10-15 15:47:18,801][52866] Updated weights for policy 1, policy_version 24770 (0.0007) -[2023-10-15 15:47:19,178][52866] Updated weights for policy 1, policy_version 24780 (0.0007) -[2023-10-15 15:47:19,544][52866] Updated weights for policy 1, policy_version 24790 (0.0008) -[2023-10-15 15:47:19,914][52866] Updated weights for policy 1, policy_version 24800 (0.0011) -[2023-10-15 15:47:20,817][52833] Updated weights for policy 0, policy_version 24710 (0.0007) -[2023-10-15 15:47:21,188][52833] Updated weights for policy 0, policy_version 24720 (0.0009) -[2023-10-15 15:47:21,555][52833] Updated weights for policy 0, policy_version 24730 (0.0010) -[2023-10-15 15:47:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 50724864. Throughput: 0: 1795.6, 1: 1797.3. Samples: 12689226. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) -[2023-10-15 15:47:23,442][51532] Avg episode reward: [(0, '34.230'), (1, '34.810')] -[2023-10-15 15:47:23,898][52866] Updated weights for policy 1, policy_version 24810 (0.0009) -[2023-10-15 15:47:24,258][52866] Updated weights for policy 1, policy_version 24820 (0.0009) -[2023-10-15 15:47:24,621][52866] Updated weights for policy 1, policy_version 24830 (0.0011) -[2023-10-15 15:47:25,361][52833] Updated weights for policy 0, policy_version 24740 (0.0011) -[2023-10-15 15:47:25,753][52833] Updated weights for policy 0, policy_version 24750 (0.0008) -[2023-10-15 15:47:26,126][52833] Updated weights for policy 0, policy_version 24760 (0.0009) -[2023-10-15 15:47:28,420][52866] Updated weights for policy 1, policy_version 24840 (0.0009) -[2023-10-15 15:47:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 50790400. Throughput: 0: 1800.4, 1: 1803.7. Samples: 12711698. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) -[2023-10-15 15:47:28,442][51532] Avg episode reward: [(0, '35.310'), (1, '33.150')] -[2023-10-15 15:47:28,786][52866] Updated weights for policy 1, policy_version 24850 (0.0007) -[2023-10-15 15:47:29,164][52866] Updated weights for policy 1, policy_version 24860 (0.0007) -[2023-10-15 15:47:29,845][52833] Updated weights for policy 0, policy_version 24770 (0.0009) -[2023-10-15 15:47:30,216][52833] Updated weights for policy 0, policy_version 24780 (0.0009) -[2023-10-15 15:47:30,580][52833] Updated weights for policy 0, policy_version 24790 (0.0011) -[2023-10-15 15:47:30,955][52833] Updated weights for policy 0, policy_version 24800 (0.0010) -[2023-10-15 15:47:32,886][52866] Updated weights for policy 1, policy_version 24870 (0.0007) -[2023-10-15 15:47:33,253][52866] Updated weights for policy 1, policy_version 24880 (0.0009) -[2023-10-15 15:47:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 50855936. Throughput: 0: 1804.4, 1: 1792.4. Samples: 12721758. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) -[2023-10-15 15:47:33,442][51532] Avg episode reward: [(0, '34.220'), (1, '34.460')] -[2023-10-15 15:47:33,605][52866] Updated weights for policy 1, policy_version 24890 (0.0010) -[2023-10-15 15:47:34,590][52833] Updated weights for policy 0, policy_version 24810 (0.0007) -[2023-10-15 15:47:34,962][52833] Updated weights for policy 0, policy_version 24820 (0.0008) -[2023-10-15 15:47:35,330][52833] Updated weights for policy 0, policy_version 24830 (0.0007) -[2023-10-15 15:47:37,334][52866] Updated weights for policy 1, policy_version 24900 (0.0009) -[2023-10-15 15:47:37,705][52866] Updated weights for policy 1, policy_version 24910 (0.0007) -[2023-10-15 15:47:38,068][52866] Updated weights for policy 1, policy_version 24920 (0.0007) -[2023-10-15 15:47:38,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 50954240. Throughput: 0: 1806.9, 1: 1799.3. Samples: 12743994. Policy #0 lag: (min: 3.0, avg: 9.6, max: 35.0) -[2023-10-15 15:47:38,442][51532] Avg episode reward: [(0, '33.750'), (1, '33.930')] -[2023-10-15 15:47:39,046][52833] Updated weights for policy 0, policy_version 24840 (0.0010) -[2023-10-15 15:47:39,419][52833] Updated weights for policy 0, policy_version 24850 (0.0010) -[2023-10-15 15:47:39,784][52833] Updated weights for policy 0, policy_version 24860 (0.0010) -[2023-10-15 15:47:41,869][52866] Updated weights for policy 1, policy_version 24930 (0.0007) -[2023-10-15 15:47:42,238][52866] Updated weights for policy 1, policy_version 24940 (0.0010) -[2023-10-15 15:47:42,592][52866] Updated weights for policy 1, policy_version 24950 (0.0008) -[2023-10-15 15:47:42,955][52866] Updated weights for policy 1, policy_version 24960 (0.0008) -[2023-10-15 15:47:43,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 51019776. Throughput: 0: 1806.1, 1: 1789.9. Samples: 12765234. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) -[2023-10-15 15:47:43,442][51532] Avg episode reward: [(0, '32.550'), (1, '36.590')] -[2023-10-15 15:47:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000024960_25559040.pth... -[2023-10-15 15:47:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000023296_23855104.pth -[2023-10-15 15:47:43,493][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000024960_25559040.pth -[2023-10-15 15:47:43,616][52833] Updated weights for policy 0, policy_version 24870 (0.0010) -[2023-10-15 15:47:43,980][52833] Updated weights for policy 0, policy_version 24880 (0.0008) -[2023-10-15 15:47:44,344][52833] Updated weights for policy 0, policy_version 24890 (0.0007) -[2023-10-15 15:47:44,563][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000024896_25493504.pth... -[2023-10-15 15:47:44,602][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000023200_23756800.pth -[2023-10-15 15:47:44,608][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000024896_25493504.pth -[2023-10-15 15:47:46,748][52866] Updated weights for policy 1, policy_version 24970 (0.0009) -[2023-10-15 15:47:47,122][52866] Updated weights for policy 1, policy_version 24980 (0.0008) -[2023-10-15 15:47:47,479][52866] Updated weights for policy 1, policy_version 24990 (0.0008) -[2023-10-15 15:47:48,040][52833] Updated weights for policy 0, policy_version 24900 (0.0008) -[2023-10-15 15:47:48,418][52833] Updated weights for policy 0, policy_version 24910 (0.0010) -[2023-10-15 15:47:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 51085312. Throughput: 0: 1803.2, 1: 1803.5. Samples: 12776480. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) -[2023-10-15 15:47:48,441][51532] Avg episode reward: [(0, '29.700'), (1, '38.570')] -[2023-10-15 15:47:48,442][52518] Saving new best policy, reward=38.570! -[2023-10-15 15:47:48,779][52833] Updated weights for policy 0, policy_version 24920 (0.0008) -[2023-10-15 15:47:51,206][52866] Updated weights for policy 1, policy_version 25000 (0.0009) -[2023-10-15 15:47:51,574][52866] Updated weights for policy 1, policy_version 25010 (0.0008) -[2023-10-15 15:47:51,939][52866] Updated weights for policy 1, policy_version 25020 (0.0009) -[2023-10-15 15:47:52,471][52833] Updated weights for policy 0, policy_version 24930 (0.0007) -[2023-10-15 15:47:52,843][52833] Updated weights for policy 0, policy_version 24940 (0.0008) -[2023-10-15 15:47:53,211][52833] Updated weights for policy 0, policy_version 24950 (0.0008) -[2023-10-15 15:47:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 51150848. Throughput: 0: 1801.7, 1: 1793.0. Samples: 12797760. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) -[2023-10-15 15:47:53,441][51532] Avg episode reward: [(0, '30.530'), (1, '38.910')] -[2023-10-15 15:47:53,442][52518] Saving new best policy, reward=38.910! -[2023-10-15 15:47:53,578][52833] Updated weights for policy 0, policy_version 24960 (0.0010) -[2023-10-15 15:47:55,651][52866] Updated weights for policy 1, policy_version 25030 (0.0009) -[2023-10-15 15:47:56,015][52866] Updated weights for policy 1, policy_version 25040 (0.0007) -[2023-10-15 15:47:56,388][52866] Updated weights for policy 1, policy_version 25050 (0.0007) -[2023-10-15 15:47:57,429][52833] Updated weights for policy 0, policy_version 24970 (0.0010) -[2023-10-15 15:47:57,793][52833] Updated weights for policy 0, policy_version 24980 (0.0007) -[2023-10-15 15:47:58,161][52833] Updated weights for policy 0, policy_version 24990 (0.0007) -[2023-10-15 15:47:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 51249152. Throughput: 0: 1810.0, 1: 1795.3. Samples: 12819034. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 15:47:58,441][51532] Avg episode reward: [(0, '29.240'), (1, '38.130')] -[2023-10-15 15:48:00,090][52866] Updated weights for policy 1, policy_version 25060 (0.0008) -[2023-10-15 15:48:00,450][52866] Updated weights for policy 1, policy_version 25070 (0.0008) -[2023-10-15 15:48:00,821][52866] Updated weights for policy 1, policy_version 25080 (0.0008) -[2023-10-15 15:48:01,883][52833] Updated weights for policy 0, policy_version 25000 (0.0010) -[2023-10-15 15:48:02,262][52833] Updated weights for policy 0, policy_version 25010 (0.0011) -[2023-10-15 15:48:02,635][52833] Updated weights for policy 0, policy_version 25020 (0.0007) -[2023-10-15 15:48:03,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 51314688. Throughput: 0: 1796.9, 1: 1806.3. Samples: 12830206. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 15:48:03,442][51532] Avg episode reward: [(0, '29.060'), (1, '39.160')] -[2023-10-15 15:48:03,443][52518] Saving new best policy, reward=39.160! -[2023-10-15 15:48:04,316][52866] Updated weights for policy 1, policy_version 25090 (0.0008) -[2023-10-15 15:48:04,681][52866] Updated weights for policy 1, policy_version 25100 (0.0008) -[2023-10-15 15:48:05,038][52866] Updated weights for policy 1, policy_version 25110 (0.0008) -[2023-10-15 15:48:05,410][52866] Updated weights for policy 1, policy_version 25120 (0.0009) -[2023-10-15 15:48:06,360][52833] Updated weights for policy 0, policy_version 25030 (0.0008) -[2023-10-15 15:48:06,726][52833] Updated weights for policy 0, policy_version 25040 (0.0009) -[2023-10-15 15:48:07,096][52833] Updated weights for policy 0, policy_version 25050 (0.0008) -[2023-10-15 15:48:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 51380224. Throughput: 0: 1809.3, 1: 1805.5. Samples: 12851890. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 15:48:08,442][51532] Avg episode reward: [(0, '28.270'), (1, '39.630')] -[2023-10-15 15:48:08,444][52518] Saving new best policy, reward=39.630! -[2023-10-15 15:48:09,178][52866] Updated weights for policy 1, policy_version 25130 (0.0008) -[2023-10-15 15:48:09,537][52866] Updated weights for policy 1, policy_version 25140 (0.0010) -[2023-10-15 15:48:09,907][52866] Updated weights for policy 1, policy_version 25150 (0.0011) -[2023-10-15 15:48:11,071][52833] Updated weights for policy 0, policy_version 25060 (0.0009) -[2023-10-15 15:48:11,471][52833] Updated weights for policy 0, policy_version 25070 (0.0010) -[2023-10-15 15:48:11,838][52833] Updated weights for policy 0, policy_version 25080 (0.0008) -[2023-10-15 15:48:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 51445760. Throughput: 0: 1786.5, 1: 1808.7. Samples: 12873478. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 15:48:13,441][51532] Avg episode reward: [(0, '30.150'), (1, '37.940')] -[2023-10-15 15:48:13,691][52866] Updated weights for policy 1, policy_version 25160 (0.0008) -[2023-10-15 15:48:14,059][52866] Updated weights for policy 1, policy_version 25170 (0.0007) -[2023-10-15 15:48:14,432][52866] Updated weights for policy 1, policy_version 25180 (0.0008) -[2023-10-15 15:48:15,513][52833] Updated weights for policy 0, policy_version 25090 (0.0007) -[2023-10-15 15:48:15,888][52833] Updated weights for policy 0, policy_version 25100 (0.0007) -[2023-10-15 15:48:16,246][52833] Updated weights for policy 0, policy_version 25110 (0.0007) -[2023-10-15 15:48:16,611][52833] Updated weights for policy 0, policy_version 25120 (0.0007) -[2023-10-15 15:48:17,978][52866] Updated weights for policy 1, policy_version 25190 (0.0007) -[2023-10-15 15:48:18,349][52866] Updated weights for policy 1, policy_version 25200 (0.0008) -[2023-10-15 15:48:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 51511296. Throughput: 0: 1805.4, 1: 1810.0. Samples: 12884454. Policy #0 lag: (min: 10.0, avg: 18.6, max: 42.0) -[2023-10-15 15:48:18,442][51532] Avg episode reward: [(0, '30.360'), (1, '39.720')] -[2023-10-15 15:48:18,710][52866] Updated weights for policy 1, policy_version 25210 (0.0008) -[2023-10-15 15:48:18,927][52518] Saving new best policy, reward=39.720! -[2023-10-15 15:48:20,518][52833] Updated weights for policy 0, policy_version 25130 (0.0010) -[2023-10-15 15:48:20,883][52833] Updated weights for policy 0, policy_version 25140 (0.0009) -[2023-10-15 15:48:21,260][52833] Updated weights for policy 0, policy_version 25150 (0.0009) -[2023-10-15 15:48:22,726][52866] Updated weights for policy 1, policy_version 25220 (0.0009) -[2023-10-15 15:48:23,099][52866] Updated weights for policy 1, policy_version 25230 (0.0009) -[2023-10-15 15:48:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 51576832. Throughput: 0: 1785.7, 1: 1814.3. Samples: 12905998. Policy #0 lag: (min: 10.0, avg: 18.6, max: 42.0) -[2023-10-15 15:48:23,442][51532] Avg episode reward: [(0, '32.340'), (1, '36.240')] -[2023-10-15 15:48:23,461][52866] Updated weights for policy 1, policy_version 25240 (0.0008) -[2023-10-15 15:48:24,962][52833] Updated weights for policy 0, policy_version 25160 (0.0008) -[2023-10-15 15:48:25,326][52833] Updated weights for policy 0, policy_version 25170 (0.0007) -[2023-10-15 15:48:25,698][52833] Updated weights for policy 0, policy_version 25180 (0.0010) -[2023-10-15 15:48:27,193][52866] Updated weights for policy 1, policy_version 25250 (0.0008) -[2023-10-15 15:48:27,564][52866] Updated weights for policy 1, policy_version 25260 (0.0008) -[2023-10-15 15:48:27,927][52866] Updated weights for policy 1, policy_version 25270 (0.0007) -[2023-10-15 15:48:28,305][52866] Updated weights for policy 1, policy_version 25280 (0.0008) -[2023-10-15 15:48:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 51675136. Throughput: 0: 1785.9, 1: 1820.8. Samples: 12927534. Policy #0 lag: (min: 10.0, avg: 18.6, max: 42.0) -[2023-10-15 15:48:28,442][51532] Avg episode reward: [(0, '30.130'), (1, '34.580')] -[2023-10-15 15:48:29,426][52833] Updated weights for policy 0, policy_version 25190 (0.0008) -[2023-10-15 15:48:29,794][52833] Updated weights for policy 0, policy_version 25200 (0.0008) -[2023-10-15 15:48:30,164][52833] Updated weights for policy 0, policy_version 25210 (0.0008) -[2023-10-15 15:48:31,900][52866] Updated weights for policy 1, policy_version 25290 (0.0008) -[2023-10-15 15:48:32,267][52866] Updated weights for policy 1, policy_version 25300 (0.0010) -[2023-10-15 15:48:32,640][52866] Updated weights for policy 1, policy_version 25310 (0.0010) -[2023-10-15 15:48:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 51740672. Throughput: 0: 1788.3, 1: 1811.3. Samples: 12938460. Policy #0 lag: (min: 10.0, avg: 18.6, max: 42.0) -[2023-10-15 15:48:33,442][51532] Avg episode reward: [(0, '32.080'), (1, '34.150')] -[2023-10-15 15:48:33,681][52833] Updated weights for policy 0, policy_version 25220 (0.0008) -[2023-10-15 15:48:34,048][52833] Updated weights for policy 0, policy_version 25230 (0.0009) -[2023-10-15 15:48:34,423][52833] Updated weights for policy 0, policy_version 25240 (0.0008) -[2023-10-15 15:48:36,269][52866] Updated weights for policy 1, policy_version 25320 (0.0008) -[2023-10-15 15:48:36,636][52866] Updated weights for policy 1, policy_version 25330 (0.0008) -[2023-10-15 15:48:37,004][52866] Updated weights for policy 1, policy_version 25340 (0.0007) -[2023-10-15 15:48:38,305][52833] Updated weights for policy 0, policy_version 25250 (0.0009) -[2023-10-15 15:48:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 51806208. Throughput: 0: 1787.7, 1: 1811.4. Samples: 12959722. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) -[2023-10-15 15:48:38,442][51532] Avg episode reward: [(0, '32.130'), (1, '32.370')] -[2023-10-15 15:48:38,689][52833] Updated weights for policy 0, policy_version 25260 (0.0011) -[2023-10-15 15:48:39,063][52833] Updated weights for policy 0, policy_version 25270 (0.0010) -[2023-10-15 15:48:39,439][52833] Updated weights for policy 0, policy_version 25280 (0.0010) -[2023-10-15 15:48:40,612][52866] Updated weights for policy 1, policy_version 25350 (0.0008) -[2023-10-15 15:48:40,979][52866] Updated weights for policy 1, policy_version 25360 (0.0009) -[2023-10-15 15:48:41,343][52866] Updated weights for policy 1, policy_version 25370 (0.0010) -[2023-10-15 15:48:43,145][52833] Updated weights for policy 0, policy_version 25290 (0.0009) -[2023-10-15 15:48:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 51871744. Throughput: 0: 1807.9, 1: 1819.3. Samples: 12982262. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) -[2023-10-15 15:48:43,442][51532] Avg episode reward: [(0, '33.560'), (1, '32.870')] -[2023-10-15 15:48:43,521][52833] Updated weights for policy 0, policy_version 25300 (0.0007) -[2023-10-15 15:48:43,890][52833] Updated weights for policy 0, policy_version 25310 (0.0008) -[2023-10-15 15:48:44,922][52866] Updated weights for policy 1, policy_version 25380 (0.0009) -[2023-10-15 15:48:45,289][52866] Updated weights for policy 1, policy_version 25390 (0.0010) -[2023-10-15 15:48:45,659][52866] Updated weights for policy 1, policy_version 25400 (0.0008) -[2023-10-15 15:48:47,488][52833] Updated weights for policy 0, policy_version 25320 (0.0009) -[2023-10-15 15:48:47,861][52833] Updated weights for policy 0, policy_version 25330 (0.0009) -[2023-10-15 15:48:48,223][52833] Updated weights for policy 0, policy_version 25340 (0.0010) -[2023-10-15 15:48:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 51970048. Throughput: 0: 1790.3, 1: 1812.5. Samples: 12992334. Policy #0 lag: (min: 25.0, avg: 39.9, max: 57.0) -[2023-10-15 15:48:48,441][51532] Avg episode reward: [(0, '30.580'), (1, '32.880')] -[2023-10-15 15:48:49,490][52866] Updated weights for policy 1, policy_version 25410 (0.0007) -[2023-10-15 15:48:49,868][52866] Updated weights for policy 1, policy_version 25420 (0.0010) -[2023-10-15 15:48:50,232][52866] Updated weights for policy 1, policy_version 25430 (0.0009) -[2023-10-15 15:48:50,594][52866] Updated weights for policy 1, policy_version 25440 (0.0009) -[2023-10-15 15:48:52,072][52833] Updated weights for policy 0, policy_version 25350 (0.0007) -[2023-10-15 15:48:52,442][52833] Updated weights for policy 0, policy_version 25360 (0.0008) -[2023-10-15 15:48:52,816][52833] Updated weights for policy 0, policy_version 25370 (0.0007) -[2023-10-15 15:48:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 52035584. Throughput: 0: 1805.8, 1: 1803.7. Samples: 13014318. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) -[2023-10-15 15:48:53,442][51532] Avg episode reward: [(0, '29.690'), (1, '35.580')] -[2023-10-15 15:48:54,567][52866] Updated weights for policy 1, policy_version 25450 (0.0007) -[2023-10-15 15:48:54,946][52866] Updated weights for policy 1, policy_version 25460 (0.0008) -[2023-10-15 15:48:55,308][52866] Updated weights for policy 1, policy_version 25470 (0.0009) -[2023-10-15 15:48:56,603][52833] Updated weights for policy 0, policy_version 25380 (0.0007) -[2023-10-15 15:48:56,993][52833] Updated weights for policy 0, policy_version 25390 (0.0007) -[2023-10-15 15:48:57,361][52833] Updated weights for policy 0, policy_version 25400 (0.0008) -[2023-10-15 15:48:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 52101120. Throughput: 0: 1789.4, 1: 1799.1. Samples: 13034960. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) -[2023-10-15 15:48:58,441][51532] Avg episode reward: [(0, '31.250'), (1, '34.210')] -[2023-10-15 15:48:59,129][52866] Updated weights for policy 1, policy_version 25480 (0.0008) -[2023-10-15 15:48:59,488][52866] Updated weights for policy 1, policy_version 25490 (0.0007) -[2023-10-15 15:48:59,853][52866] Updated weights for policy 1, policy_version 25500 (0.0008) -[2023-10-15 15:49:01,016][52833] Updated weights for policy 0, policy_version 25410 (0.0010) -[2023-10-15 15:49:01,389][52833] Updated weights for policy 0, policy_version 25420 (0.0008) -[2023-10-15 15:49:01,761][52833] Updated weights for policy 0, policy_version 25430 (0.0009) -[2023-10-15 15:49:02,120][52833] Updated weights for policy 0, policy_version 25440 (0.0007) -[2023-10-15 15:49:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 52166656. Throughput: 0: 1804.0, 1: 1792.5. Samples: 13046294. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) -[2023-10-15 15:49:03,441][51532] Avg episode reward: [(0, '32.640'), (1, '34.940')] -[2023-10-15 15:49:03,622][52866] Updated weights for policy 1, policy_version 25510 (0.0007) -[2023-10-15 15:49:03,989][52866] Updated weights for policy 1, policy_version 25520 (0.0009) -[2023-10-15 15:49:04,358][52866] Updated weights for policy 1, policy_version 25530 (0.0009) -[2023-10-15 15:49:05,849][52833] Updated weights for policy 0, policy_version 25450 (0.0007) -[2023-10-15 15:49:06,216][52833] Updated weights for policy 0, policy_version 25460 (0.0009) -[2023-10-15 15:49:06,579][52833] Updated weights for policy 0, policy_version 25470 (0.0010) -[2023-10-15 15:49:08,147][52866] Updated weights for policy 1, policy_version 25540 (0.0007) -[2023-10-15 15:49:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 52232192. Throughput: 0: 1791.9, 1: 1795.7. Samples: 13067436. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) -[2023-10-15 15:49:08,441][51532] Avg episode reward: [(0, '32.450'), (1, '34.210')] -[2023-10-15 15:49:08,508][52866] Updated weights for policy 1, policy_version 25550 (0.0007) -[2023-10-15 15:49:08,873][52866] Updated weights for policy 1, policy_version 25560 (0.0007) -[2023-10-15 15:49:10,460][52833] Updated weights for policy 0, policy_version 25480 (0.0007) -[2023-10-15 15:49:10,825][52833] Updated weights for policy 0, policy_version 25490 (0.0008) -[2023-10-15 15:49:11,193][52833] Updated weights for policy 0, policy_version 25500 (0.0010) -[2023-10-15 15:49:12,642][52866] Updated weights for policy 1, policy_version 25570 (0.0008) -[2023-10-15 15:49:13,007][52866] Updated weights for policy 1, policy_version 25580 (0.0008) -[2023-10-15 15:49:13,366][52866] Updated weights for policy 1, policy_version 25590 (0.0009) -[2023-10-15 15:49:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 52297728. Throughput: 0: 1790.3, 1: 1808.6. Samples: 13089484. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:49:13,442][51532] Avg episode reward: [(0, '33.170'), (1, '34.970')] -[2023-10-15 15:49:13,736][52866] Updated weights for policy 1, policy_version 25600 (0.0007) -[2023-10-15 15:49:15,069][52833] Updated weights for policy 0, policy_version 25510 (0.0011) -[2023-10-15 15:49:15,441][52833] Updated weights for policy 0, policy_version 25520 (0.0010) -[2023-10-15 15:49:15,810][52833] Updated weights for policy 0, policy_version 25530 (0.0010) -[2023-10-15 15:49:17,552][52866] Updated weights for policy 1, policy_version 25610 (0.0008) -[2023-10-15 15:49:17,929][52866] Updated weights for policy 1, policy_version 25620 (0.0008) -[2023-10-15 15:49:18,291][52866] Updated weights for policy 1, policy_version 25630 (0.0009) -[2023-10-15 15:49:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 52396032. Throughput: 0: 1792.3, 1: 1791.5. Samples: 13099730. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:49:18,441][51532] Avg episode reward: [(0, '34.690'), (1, '35.850')] -[2023-10-15 15:49:19,549][52833] Updated weights for policy 0, policy_version 25540 (0.0010) -[2023-10-15 15:49:19,920][52833] Updated weights for policy 0, policy_version 25550 (0.0011) -[2023-10-15 15:49:20,288][52833] Updated weights for policy 0, policy_version 25560 (0.0011) -[2023-10-15 15:49:22,197][52866] Updated weights for policy 1, policy_version 25640 (0.0008) -[2023-10-15 15:49:22,561][52866] Updated weights for policy 1, policy_version 25650 (0.0008) -[2023-10-15 15:49:22,931][52866] Updated weights for policy 1, policy_version 25660 (0.0008) -[2023-10-15 15:49:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 52461568. Throughput: 0: 1777.3, 1: 1811.1. Samples: 13121202. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:49:23,441][51532] Avg episode reward: [(0, '35.520'), (1, '34.020')] -[2023-10-15 15:49:24,240][52833] Updated weights for policy 0, policy_version 25570 (0.0009) -[2023-10-15 15:49:24,613][52833] Updated weights for policy 0, policy_version 25580 (0.0007) -[2023-10-15 15:49:24,985][52833] Updated weights for policy 0, policy_version 25590 (0.0007) -[2023-10-15 15:49:25,357][52833] Updated weights for policy 0, policy_version 25600 (0.0008) -[2023-10-15 15:49:26,789][52866] Updated weights for policy 1, policy_version 25670 (0.0007) -[2023-10-15 15:49:27,149][52866] Updated weights for policy 1, policy_version 25680 (0.0008) -[2023-10-15 15:49:27,521][52866] Updated weights for policy 1, policy_version 25690 (0.0007) -[2023-10-15 15:49:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 52527104. Throughput: 0: 1783.6, 1: 1770.5. Samples: 13142200. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 15:49:28,442][51532] Avg episode reward: [(0, '36.870'), (1, '33.180')] -[2023-10-15 15:49:28,454][52410] Saving new best policy, reward=36.870! -[2023-10-15 15:49:29,116][52833] Updated weights for policy 0, policy_version 25610 (0.0010) -[2023-10-15 15:49:29,487][52833] Updated weights for policy 0, policy_version 25620 (0.0009) -[2023-10-15 15:49:29,865][52833] Updated weights for policy 0, policy_version 25630 (0.0011) -[2023-10-15 15:49:31,185][52866] Updated weights for policy 1, policy_version 25700 (0.0009) -[2023-10-15 15:49:31,552][52866] Updated weights for policy 1, policy_version 25710 (0.0007) -[2023-10-15 15:49:31,926][52866] Updated weights for policy 1, policy_version 25720 (0.0007) -[2023-10-15 15:49:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 52592640. Throughput: 0: 1777.6, 1: 1805.6. Samples: 13153578. Policy #0 lag: (min: 17.0, avg: 31.2, max: 49.0) -[2023-10-15 15:49:33,441][51532] Avg episode reward: [(0, '37.490'), (1, '30.110')] -[2023-10-15 15:49:33,615][52833] Updated weights for policy 0, policy_version 25640 (0.0011) -[2023-10-15 15:49:33,983][52833] Updated weights for policy 0, policy_version 25650 (0.0008) -[2023-10-15 15:49:34,353][52833] Updated weights for policy 0, policy_version 25660 (0.0008) -[2023-10-15 15:49:34,499][52410] Saving new best policy, reward=37.490! -[2023-10-15 15:49:35,498][52866] Updated weights for policy 1, policy_version 25730 (0.0010) -[2023-10-15 15:49:35,867][52866] Updated weights for policy 1, policy_version 25740 (0.0009) -[2023-10-15 15:49:36,231][52866] Updated weights for policy 1, policy_version 25750 (0.0010) -[2023-10-15 15:49:36,607][52866] Updated weights for policy 1, policy_version 25760 (0.0012) -[2023-10-15 15:49:37,982][52833] Updated weights for policy 0, policy_version 25670 (0.0008) -[2023-10-15 15:49:38,352][52833] Updated weights for policy 0, policy_version 25680 (0.0007) -[2023-10-15 15:49:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 52658176. Throughput: 0: 1782.4, 1: 1782.4. Samples: 13174734. Policy #0 lag: (min: 17.0, avg: 31.2, max: 49.0) -[2023-10-15 15:49:38,442][51532] Avg episode reward: [(0, '35.590'), (1, '32.320')] -[2023-10-15 15:49:38,727][52833] Updated weights for policy 0, policy_version 25690 (0.0008) -[2023-10-15 15:49:40,434][52866] Updated weights for policy 1, policy_version 25770 (0.0008) -[2023-10-15 15:49:40,799][52866] Updated weights for policy 1, policy_version 25780 (0.0008) -[2023-10-15 15:49:41,159][52866] Updated weights for policy 1, policy_version 25790 (0.0007) -[2023-10-15 15:49:42,477][52833] Updated weights for policy 0, policy_version 25700 (0.0009) -[2023-10-15 15:49:42,864][52833] Updated weights for policy 0, policy_version 25710 (0.0008) -[2023-10-15 15:49:43,237][52833] Updated weights for policy 0, policy_version 25720 (0.0009) -[2023-10-15 15:49:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 52723712. Throughput: 0: 1802.4, 1: 1788.3. Samples: 13196540. Policy #0 lag: (min: 17.0, avg: 31.2, max: 49.0) -[2023-10-15 15:49:43,442][51532] Avg episode reward: [(0, '35.610'), (1, '32.630')] -[2023-10-15 15:49:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000025792_26411008.pth... -[2023-10-15 15:49:43,484][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000024128_24707072.pth -[2023-10-15 15:49:43,525][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000025728_26345472.pth... -[2023-10-15 15:49:43,563][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000024064_24641536.pth -[2023-10-15 15:49:44,835][52866] Updated weights for policy 1, policy_version 25800 (0.0008) -[2023-10-15 15:49:45,206][52866] Updated weights for policy 1, policy_version 25810 (0.0007) -[2023-10-15 15:49:45,564][52866] Updated weights for policy 1, policy_version 25820 (0.0008) -[2023-10-15 15:49:46,938][52833] Updated weights for policy 0, policy_version 25730 (0.0008) -[2023-10-15 15:49:47,307][52833] Updated weights for policy 0, policy_version 25740 (0.0008) -[2023-10-15 15:49:47,673][52833] Updated weights for policy 0, policy_version 25750 (0.0008) -[2023-10-15 15:49:48,035][52833] Updated weights for policy 0, policy_version 25760 (0.0009) -[2023-10-15 15:49:48,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 52822016. Throughput: 0: 1779.2, 1: 1794.3. Samples: 13207102. Policy #0 lag: (min: 17.0, avg: 25.2, max: 49.0) -[2023-10-15 15:49:48,441][51532] Avg episode reward: [(0, '35.220'), (1, '32.320')] -[2023-10-15 15:49:49,310][52866] Updated weights for policy 1, policy_version 25830 (0.0008) -[2023-10-15 15:49:49,677][52866] Updated weights for policy 1, policy_version 25840 (0.0008) -[2023-10-15 15:49:50,048][52866] Updated weights for policy 1, policy_version 25850 (0.0009) -[2023-10-15 15:49:51,930][52833] Updated weights for policy 0, policy_version 25770 (0.0007) -[2023-10-15 15:49:52,308][52833] Updated weights for policy 0, policy_version 25780 (0.0008) -[2023-10-15 15:49:52,677][52833] Updated weights for policy 0, policy_version 25790 (0.0007) -[2023-10-15 15:49:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 52887552. Throughput: 0: 1802.3, 1: 1788.9. Samples: 13229044. Policy #0 lag: (min: 17.0, avg: 25.2, max: 49.0) -[2023-10-15 15:49:53,443][51532] Avg episode reward: [(0, '32.000'), (1, '31.730')] -[2023-10-15 15:49:53,721][52866] Updated weights for policy 1, policy_version 25860 (0.0009) -[2023-10-15 15:49:54,094][52866] Updated weights for policy 1, policy_version 25870 (0.0009) -[2023-10-15 15:49:54,467][52866] Updated weights for policy 1, policy_version 25880 (0.0008) -[2023-10-15 15:49:56,350][52833] Updated weights for policy 0, policy_version 25800 (0.0009) -[2023-10-15 15:49:56,720][52833] Updated weights for policy 0, policy_version 25810 (0.0008) -[2023-10-15 15:49:57,080][52833] Updated weights for policy 0, policy_version 25820 (0.0007) -[2023-10-15 15:49:58,153][52866] Updated weights for policy 1, policy_version 25890 (0.0009) -[2023-10-15 15:49:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 52953088. Throughput: 0: 1779.2, 1: 1804.0. Samples: 13250728. Policy #0 lag: (min: 17.0, avg: 25.2, max: 49.0) -[2023-10-15 15:49:58,442][51532] Avg episode reward: [(0, '31.680'), (1, '31.910')] -[2023-10-15 15:49:58,529][52866] Updated weights for policy 1, policy_version 25900 (0.0007) -[2023-10-15 15:49:58,891][52866] Updated weights for policy 1, policy_version 25910 (0.0008) -[2023-10-15 15:49:59,265][52866] Updated weights for policy 1, policy_version 25920 (0.0008) -[2023-10-15 15:50:00,832][52833] Updated weights for policy 0, policy_version 25830 (0.0008) -[2023-10-15 15:50:01,203][52833] Updated weights for policy 0, policy_version 25840 (0.0007) -[2023-10-15 15:50:01,565][52833] Updated weights for policy 0, policy_version 25850 (0.0008) -[2023-10-15 15:50:02,832][52866] Updated weights for policy 1, policy_version 25930 (0.0008) -[2023-10-15 15:50:03,193][52866] Updated weights for policy 1, policy_version 25940 (0.0007) -[2023-10-15 15:50:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 53018624. Throughput: 0: 1808.2, 1: 1801.1. Samples: 13262148. Policy #0 lag: (min: 17.0, avg: 25.2, max: 49.0) -[2023-10-15 15:50:03,442][51532] Avg episode reward: [(0, '30.960'), (1, '32.730')] -[2023-10-15 15:50:03,570][52866] Updated weights for policy 1, policy_version 25950 (0.0008) -[2023-10-15 15:50:05,408][52833] Updated weights for policy 0, policy_version 25860 (0.0007) -[2023-10-15 15:50:05,776][52833] Updated weights for policy 0, policy_version 25870 (0.0007) -[2023-10-15 15:50:06,146][52833] Updated weights for policy 0, policy_version 25880 (0.0009) -[2023-10-15 15:50:07,290][52866] Updated weights for policy 1, policy_version 25960 (0.0009) -[2023-10-15 15:50:07,651][52866] Updated weights for policy 1, policy_version 25970 (0.0007) -[2023-10-15 15:50:08,023][52866] Updated weights for policy 1, policy_version 25980 (0.0008) -[2023-10-15 15:50:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 53116928. Throughput: 0: 1792.8, 1: 1812.2. Samples: 13283428. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:50:08,442][51532] Avg episode reward: [(0, '30.980'), (1, '34.820')] -[2023-10-15 15:50:09,779][52833] Updated weights for policy 0, policy_version 25890 (0.0008) -[2023-10-15 15:50:10,154][52833] Updated weights for policy 0, policy_version 25900 (0.0009) -[2023-10-15 15:50:10,519][52833] Updated weights for policy 0, policy_version 25910 (0.0011) -[2023-10-15 15:50:10,887][52833] Updated weights for policy 0, policy_version 25920 (0.0010) -[2023-10-15 15:50:11,682][52866] Updated weights for policy 1, policy_version 25990 (0.0009) -[2023-10-15 15:50:12,045][52866] Updated weights for policy 1, policy_version 26000 (0.0009) -[2023-10-15 15:50:12,412][52866] Updated weights for policy 1, policy_version 26010 (0.0009) -[2023-10-15 15:50:13,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 53182464. Throughput: 0: 1792.9, 1: 1816.3. Samples: 13304612. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:50:13,442][51532] Avg episode reward: [(0, '31.630'), (1, '36.260')] -[2023-10-15 15:50:14,722][52833] Updated weights for policy 0, policy_version 25930 (0.0008) -[2023-10-15 15:50:15,089][52833] Updated weights for policy 0, policy_version 25940 (0.0008) -[2023-10-15 15:50:15,468][52833] Updated weights for policy 0, policy_version 25950 (0.0009) -[2023-10-15 15:50:16,061][52866] Updated weights for policy 1, policy_version 26020 (0.0009) -[2023-10-15 15:50:16,435][52866] Updated weights for policy 1, policy_version 26030 (0.0009) -[2023-10-15 15:50:16,798][52866] Updated weights for policy 1, policy_version 26040 (0.0010) -[2023-10-15 15:50:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 53248000. Throughput: 0: 1790.4, 1: 1813.5. Samples: 13315758. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:50:18,442][51532] Avg episode reward: [(0, '30.840'), (1, '35.660')] -[2023-10-15 15:50:19,205][52833] Updated weights for policy 0, policy_version 25960 (0.0008) -[2023-10-15 15:50:19,577][52833] Updated weights for policy 0, policy_version 25970 (0.0008) -[2023-10-15 15:50:19,935][52833] Updated weights for policy 0, policy_version 25980 (0.0007) -[2023-10-15 15:50:20,423][52866] Updated weights for policy 1, policy_version 26050 (0.0010) -[2023-10-15 15:50:20,785][52866] Updated weights for policy 1, policy_version 26060 (0.0008) -[2023-10-15 15:50:21,163][52866] Updated weights for policy 1, policy_version 26070 (0.0009) -[2023-10-15 15:50:21,521][52866] Updated weights for policy 1, policy_version 26080 (0.0009) -[2023-10-15 15:50:23,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 53313536. Throughput: 0: 1793.5, 1: 1813.6. Samples: 13337052. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-15 15:50:23,441][51532] Avg episode reward: [(0, '30.000'), (1, '35.820')] -[2023-10-15 15:50:23,686][52833] Updated weights for policy 0, policy_version 25990 (0.0007) -[2023-10-15 15:50:24,048][52833] Updated weights for policy 0, policy_version 26000 (0.0008) -[2023-10-15 15:50:24,430][52833] Updated weights for policy 0, policy_version 26010 (0.0007) -[2023-10-15 15:50:25,448][52866] Updated weights for policy 1, policy_version 26090 (0.0009) -[2023-10-15 15:50:25,815][52866] Updated weights for policy 1, policy_version 26100 (0.0007) -[2023-10-15 15:50:26,179][52866] Updated weights for policy 1, policy_version 26110 (0.0007) -[2023-10-15 15:50:28,342][52833] Updated weights for policy 0, policy_version 26020 (0.0009) -[2023-10-15 15:50:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 53379072. Throughput: 0: 1805.2, 1: 1809.2. Samples: 13359188. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) -[2023-10-15 15:50:28,442][51532] Avg episode reward: [(0, '30.680'), (1, '35.320')] -[2023-10-15 15:50:28,734][52833] Updated weights for policy 0, policy_version 26030 (0.0008) -[2023-10-15 15:50:29,105][52833] Updated weights for policy 0, policy_version 26040 (0.0009) -[2023-10-15 15:50:29,933][52866] Updated weights for policy 1, policy_version 26120 (0.0009) -[2023-10-15 15:50:30,295][52866] Updated weights for policy 1, policy_version 26130 (0.0007) -[2023-10-15 15:50:30,651][52866] Updated weights for policy 1, policy_version 26140 (0.0007) -[2023-10-15 15:50:32,730][52833] Updated weights for policy 0, policy_version 26050 (0.0010) -[2023-10-15 15:50:33,092][52833] Updated weights for policy 0, policy_version 26060 (0.0008) -[2023-10-15 15:50:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 53444608. Throughput: 0: 1791.6, 1: 1806.7. Samples: 13369024. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) -[2023-10-15 15:50:33,442][51532] Avg episode reward: [(0, '32.490'), (1, '36.030')] -[2023-10-15 15:50:33,461][52833] Updated weights for policy 0, policy_version 26070 (0.0007) -[2023-10-15 15:50:33,818][52833] Updated weights for policy 0, policy_version 26080 (0.0008) -[2023-10-15 15:50:34,421][52866] Updated weights for policy 1, policy_version 26150 (0.0009) -[2023-10-15 15:50:34,798][52866] Updated weights for policy 1, policy_version 26160 (0.0007) -[2023-10-15 15:50:35,165][52866] Updated weights for policy 1, policy_version 26170 (0.0008) -[2023-10-15 15:50:37,515][52833] Updated weights for policy 0, policy_version 26090 (0.0010) -[2023-10-15 15:50:37,881][52833] Updated weights for policy 0, policy_version 26100 (0.0008) -[2023-10-15 15:50:38,246][52833] Updated weights for policy 0, policy_version 26110 (0.0009) -[2023-10-15 15:50:38,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 53542912. Throughput: 0: 1803.6, 1: 1809.8. Samples: 13391646. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) -[2023-10-15 15:50:38,442][51532] Avg episode reward: [(0, '32.790'), (1, '36.200')] -[2023-10-15 15:50:38,993][52866] Updated weights for policy 1, policy_version 26180 (0.0007) -[2023-10-15 15:50:39,359][52866] Updated weights for policy 1, policy_version 26190 (0.0008) -[2023-10-15 15:50:39,731][52866] Updated weights for policy 1, policy_version 26200 (0.0008) -[2023-10-15 15:50:42,107][52833] Updated weights for policy 0, policy_version 26120 (0.0009) -[2023-10-15 15:50:42,474][52833] Updated weights for policy 0, policy_version 26130 (0.0008) -[2023-10-15 15:50:42,839][52833] Updated weights for policy 0, policy_version 26140 (0.0008) -[2023-10-15 15:50:43,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 53608448. Throughput: 0: 1795.9, 1: 1806.0. Samples: 13412814. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) -[2023-10-15 15:50:43,442][51532] Avg episode reward: [(0, '33.090'), (1, '36.760')] -[2023-10-15 15:50:43,481][52866] Updated weights for policy 1, policy_version 26210 (0.0010) -[2023-10-15 15:50:43,842][52866] Updated weights for policy 1, policy_version 26220 (0.0008) -[2023-10-15 15:50:44,199][52866] Updated weights for policy 1, policy_version 26230 (0.0007) -[2023-10-15 15:50:44,568][52866] Updated weights for policy 1, policy_version 26240 (0.0009) -[2023-10-15 15:50:46,516][52833] Updated weights for policy 0, policy_version 26150 (0.0009) -[2023-10-15 15:50:46,882][52833] Updated weights for policy 0, policy_version 26160 (0.0008) -[2023-10-15 15:50:47,268][52833] Updated weights for policy 0, policy_version 26170 (0.0010) -[2023-10-15 15:50:48,270][52866] Updated weights for policy 1, policy_version 26250 (0.0009) -[2023-10-15 15:50:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 53673984. Throughput: 0: 1794.8, 1: 1800.3. Samples: 13423930. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) -[2023-10-15 15:50:48,442][51532] Avg episode reward: [(0, '33.370'), (1, '33.640')] -[2023-10-15 15:50:48,642][52866] Updated weights for policy 1, policy_version 26260 (0.0010) -[2023-10-15 15:50:49,008][52866] Updated weights for policy 1, policy_version 26270 (0.0011) -[2023-10-15 15:50:50,969][52833] Updated weights for policy 0, policy_version 26180 (0.0009) -[2023-10-15 15:50:51,336][52833] Updated weights for policy 0, policy_version 26190 (0.0011) -[2023-10-15 15:50:51,704][52833] Updated weights for policy 0, policy_version 26200 (0.0009) -[2023-10-15 15:50:52,798][52866] Updated weights for policy 1, policy_version 26280 (0.0009) -[2023-10-15 15:50:53,172][52866] Updated weights for policy 1, policy_version 26290 (0.0010) -[2023-10-15 15:50:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 53739520. Throughput: 0: 1794.2, 1: 1795.2. Samples: 13444954. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) -[2023-10-15 15:50:53,442][51532] Avg episode reward: [(0, '31.620'), (1, '33.810')] -[2023-10-15 15:50:53,547][52866] Updated weights for policy 1, policy_version 26300 (0.0008) -[2023-10-15 15:50:55,422][52833] Updated weights for policy 0, policy_version 26210 (0.0008) -[2023-10-15 15:50:55,797][52833] Updated weights for policy 0, policy_version 26220 (0.0010) -[2023-10-15 15:50:56,179][52833] Updated weights for policy 0, policy_version 26230 (0.0009) -[2023-10-15 15:50:56,541][52833] Updated weights for policy 0, policy_version 26240 (0.0008) -[2023-10-15 15:50:57,281][52866] Updated weights for policy 1, policy_version 26310 (0.0007) -[2023-10-15 15:50:57,655][52866] Updated weights for policy 1, policy_version 26320 (0.0008) -[2023-10-15 15:50:58,014][52866] Updated weights for policy 1, policy_version 26330 (0.0008) -[2023-10-15 15:50:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 53837824. Throughput: 0: 1794.2, 1: 1802.0. Samples: 13466442. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) -[2023-10-15 15:50:58,442][51532] Avg episode reward: [(0, '32.630'), (1, '35.230')] -[2023-10-15 15:51:00,178][52833] Updated weights for policy 0, policy_version 26250 (0.0010) -[2023-10-15 15:51:00,539][52833] Updated weights for policy 0, policy_version 26260 (0.0009) -[2023-10-15 15:51:00,906][52833] Updated weights for policy 0, policy_version 26270 (0.0009) -[2023-10-15 15:51:01,858][52866] Updated weights for policy 1, policy_version 26340 (0.0007) -[2023-10-15 15:51:02,221][52866] Updated weights for policy 1, policy_version 26350 (0.0008) -[2023-10-15 15:51:02,594][52866] Updated weights for policy 1, policy_version 26360 (0.0009) -[2023-10-15 15:51:03,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 53903360. Throughput: 0: 1802.6, 1: 1792.2. Samples: 13477524. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) -[2023-10-15 15:51:03,441][51532] Avg episode reward: [(0, '33.440'), (1, '36.290')] -[2023-10-15 15:51:04,636][52833] Updated weights for policy 0, policy_version 26280 (0.0010) -[2023-10-15 15:51:05,005][52833] Updated weights for policy 0, policy_version 26290 (0.0010) -[2023-10-15 15:51:05,367][52833] Updated weights for policy 0, policy_version 26300 (0.0008) -[2023-10-15 15:51:06,443][52866] Updated weights for policy 1, policy_version 26370 (0.0009) -[2023-10-15 15:51:06,810][52866] Updated weights for policy 1, policy_version 26380 (0.0008) -[2023-10-15 15:51:07,171][52866] Updated weights for policy 1, policy_version 26390 (0.0010) -[2023-10-15 15:51:07,541][52866] Updated weights for policy 1, policy_version 26400 (0.0010) -[2023-10-15 15:51:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 53968896. Throughput: 0: 1787.5, 1: 1800.3. Samples: 13498506. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) -[2023-10-15 15:51:08,442][51532] Avg episode reward: [(0, '31.640'), (1, '35.240')] -[2023-10-15 15:51:09,113][52833] Updated weights for policy 0, policy_version 26310 (0.0008) -[2023-10-15 15:51:09,487][52833] Updated weights for policy 0, policy_version 26320 (0.0007) -[2023-10-15 15:51:09,848][52833] Updated weights for policy 0, policy_version 26330 (0.0008) -[2023-10-15 15:51:11,368][52866] Updated weights for policy 1, policy_version 26410 (0.0010) -[2023-10-15 15:51:11,728][52866] Updated weights for policy 1, policy_version 26420 (0.0009) -[2023-10-15 15:51:12,104][52866] Updated weights for policy 1, policy_version 26430 (0.0010) -[2023-10-15 15:51:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 54034432. Throughput: 0: 1792.5, 1: 1785.8. Samples: 13520210. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) -[2023-10-15 15:51:13,442][51532] Avg episode reward: [(0, '30.640'), (1, '35.600')] -[2023-10-15 15:51:13,709][52833] Updated weights for policy 0, policy_version 26340 (0.0008) -[2023-10-15 15:51:14,093][52833] Updated weights for policy 0, policy_version 26350 (0.0007) -[2023-10-15 15:51:14,460][52833] Updated weights for policy 0, policy_version 26360 (0.0007) -[2023-10-15 15:51:15,704][52866] Updated weights for policy 1, policy_version 26440 (0.0008) -[2023-10-15 15:51:16,066][52866] Updated weights for policy 1, policy_version 26450 (0.0010) -[2023-10-15 15:51:16,432][52866] Updated weights for policy 1, policy_version 26460 (0.0009) -[2023-10-15 15:51:18,262][52833] Updated weights for policy 0, policy_version 26370 (0.0008) -[2023-10-15 15:51:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 54099968. Throughput: 0: 1790.2, 1: 1807.2. Samples: 13530910. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) -[2023-10-15 15:51:18,441][51532] Avg episode reward: [(0, '30.340'), (1, '34.240')] -[2023-10-15 15:51:18,628][52833] Updated weights for policy 0, policy_version 26380 (0.0008) -[2023-10-15 15:51:18,996][52833] Updated weights for policy 0, policy_version 26390 (0.0008) -[2023-10-15 15:51:19,369][52833] Updated weights for policy 0, policy_version 26400 (0.0010) -[2023-10-15 15:51:20,203][52866] Updated weights for policy 1, policy_version 26470 (0.0009) -[2023-10-15 15:51:20,577][52866] Updated weights for policy 1, policy_version 26480 (0.0008) -[2023-10-15 15:51:20,941][52866] Updated weights for policy 1, policy_version 26490 (0.0007) -[2023-10-15 15:51:23,072][52833] Updated weights for policy 0, policy_version 26410 (0.0009) -[2023-10-15 15:51:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 54165504. Throughput: 0: 1787.3, 1: 1785.8. Samples: 13552436. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) -[2023-10-15 15:51:23,442][51532] Avg episode reward: [(0, '28.920'), (1, '33.310')] -[2023-10-15 15:51:23,444][52833] Updated weights for policy 0, policy_version 26420 (0.0007) -[2023-10-15 15:51:23,834][52833] Updated weights for policy 0, policy_version 26430 (0.0011) -[2023-10-15 15:51:24,568][52866] Updated weights for policy 1, policy_version 26500 (0.0008) -[2023-10-15 15:51:24,936][52866] Updated weights for policy 1, policy_version 26510 (0.0008) -[2023-10-15 15:51:25,306][52866] Updated weights for policy 1, policy_version 26520 (0.0008) -[2023-10-15 15:51:27,478][52833] Updated weights for policy 0, policy_version 26440 (0.0010) -[2023-10-15 15:51:27,843][52833] Updated weights for policy 0, policy_version 26450 (0.0009) -[2023-10-15 15:51:28,222][52833] Updated weights for policy 0, policy_version 26460 (0.0008) -[2023-10-15 15:51:28,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 54263808. Throughput: 0: 1798.1, 1: 1785.6. Samples: 13574080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:51:28,442][51532] Avg episode reward: [(0, '29.500'), (1, '32.850')] -[2023-10-15 15:51:29,147][52866] Updated weights for policy 1, policy_version 26530 (0.0008) -[2023-10-15 15:51:29,513][52866] Updated weights for policy 1, policy_version 26540 (0.0007) -[2023-10-15 15:51:29,889][52866] Updated weights for policy 1, policy_version 26550 (0.0009) -[2023-10-15 15:51:30,258][52866] Updated weights for policy 1, policy_version 26560 (0.0008) -[2023-10-15 15:51:32,019][52833] Updated weights for policy 0, policy_version 26470 (0.0008) -[2023-10-15 15:51:32,390][52833] Updated weights for policy 0, policy_version 26480 (0.0007) -[2023-10-15 15:51:32,763][52833] Updated weights for policy 0, policy_version 26490 (0.0007) -[2023-10-15 15:51:33,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 54329344. Throughput: 0: 1788.5, 1: 1785.7. Samples: 13584768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:51:33,442][51532] Avg episode reward: [(0, '30.930'), (1, '30.820')] -[2023-10-15 15:51:33,978][52866] Updated weights for policy 1, policy_version 26570 (0.0007) -[2023-10-15 15:51:34,350][52866] Updated weights for policy 1, policy_version 26580 (0.0007) -[2023-10-15 15:51:34,722][52866] Updated weights for policy 1, policy_version 26590 (0.0011) -[2023-10-15 15:51:36,599][52833] Updated weights for policy 0, policy_version 26500 (0.0010) -[2023-10-15 15:51:36,969][52833] Updated weights for policy 0, policy_version 26510 (0.0009) -[2023-10-15 15:51:37,328][52833] Updated weights for policy 0, policy_version 26520 (0.0007) -[2023-10-15 15:51:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54394880. Throughput: 0: 1798.7, 1: 1789.5. Samples: 13606424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:51:38,441][51532] Avg episode reward: [(0, '32.850'), (1, '28.890')] -[2023-10-15 15:51:38,483][52866] Updated weights for policy 1, policy_version 26600 (0.0009) -[2023-10-15 15:51:38,851][52866] Updated weights for policy 1, policy_version 26610 (0.0009) -[2023-10-15 15:51:39,224][52866] Updated weights for policy 1, policy_version 26620 (0.0009) -[2023-10-15 15:51:41,007][52833] Updated weights for policy 0, policy_version 26530 (0.0010) -[2023-10-15 15:51:41,377][52833] Updated weights for policy 0, policy_version 26540 (0.0008) -[2023-10-15 15:51:41,746][52833] Updated weights for policy 0, policy_version 26550 (0.0007) -[2023-10-15 15:51:42,119][52833] Updated weights for policy 0, policy_version 26560 (0.0009) -[2023-10-15 15:51:42,831][52866] Updated weights for policy 1, policy_version 26630 (0.0008) -[2023-10-15 15:51:43,199][52866] Updated weights for policy 1, policy_version 26640 (0.0007) -[2023-10-15 15:51:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 54460416. Throughput: 0: 1778.2, 1: 1804.7. Samples: 13627674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:51:43,442][51532] Avg episode reward: [(0, '30.940'), (1, '30.660')] -[2023-10-15 15:51:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000026560_27197440.pth... -[2023-10-15 15:51:43,486][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000024896_25493504.pth -[2023-10-15 15:51:43,563][52866] Updated weights for policy 1, policy_version 26650 (0.0007) -[2023-10-15 15:51:43,782][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000026656_27295744.pth... -[2023-10-15 15:51:43,822][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000024960_25559040.pth -[2023-10-15 15:51:45,963][52833] Updated weights for policy 0, policy_version 26570 (0.0008) -[2023-10-15 15:51:46,330][52833] Updated weights for policy 0, policy_version 26580 (0.0008) -[2023-10-15 15:51:46,695][52833] Updated weights for policy 0, policy_version 26590 (0.0008) -[2023-10-15 15:51:47,292][52866] Updated weights for policy 1, policy_version 26660 (0.0011) -[2023-10-15 15:51:47,659][52866] Updated weights for policy 1, policy_version 26670 (0.0008) -[2023-10-15 15:51:48,028][52866] Updated weights for policy 1, policy_version 26680 (0.0010) -[2023-10-15 15:51:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 54558720. Throughput: 0: 1804.8, 1: 1789.8. Samples: 13639280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:51:48,441][51532] Avg episode reward: [(0, '33.020'), (1, '29.550')] -[2023-10-15 15:51:50,677][52833] Updated weights for policy 0, policy_version 26600 (0.0007) -[2023-10-15 15:51:51,055][52833] Updated weights for policy 0, policy_version 26610 (0.0007) -[2023-10-15 15:51:51,423][52833] Updated weights for policy 0, policy_version 26620 (0.0009) -[2023-10-15 15:51:51,954][52866] Updated weights for policy 1, policy_version 26690 (0.0010) -[2023-10-15 15:51:52,318][52866] Updated weights for policy 1, policy_version 26700 (0.0008) -[2023-10-15 15:51:52,697][52866] Updated weights for policy 1, policy_version 26710 (0.0008) -[2023-10-15 15:51:53,068][52866] Updated weights for policy 1, policy_version 26720 (0.0008) -[2023-10-15 15:51:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 54624256. Throughput: 0: 1789.2, 1: 1807.4. Samples: 13660352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:51:53,442][51532] Avg episode reward: [(0, '33.200'), (1, '31.270')] -[2023-10-15 15:51:55,130][52833] Updated weights for policy 0, policy_version 26630 (0.0008) -[2023-10-15 15:51:55,497][52833] Updated weights for policy 0, policy_version 26640 (0.0007) -[2023-10-15 15:51:55,860][52833] Updated weights for policy 0, policy_version 26650 (0.0007) -[2023-10-15 15:51:56,700][52866] Updated weights for policy 1, policy_version 26730 (0.0007) -[2023-10-15 15:51:57,070][52866] Updated weights for policy 1, policy_version 26740 (0.0007) -[2023-10-15 15:51:57,442][52866] Updated weights for policy 1, policy_version 26750 (0.0008) -[2023-10-15 15:51:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 54689792. Throughput: 0: 1787.6, 1: 1800.1. Samples: 13681656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:51:58,442][51532] Avg episode reward: [(0, '33.350'), (1, '33.180')] -[2023-10-15 15:51:59,568][52833] Updated weights for policy 0, policy_version 26660 (0.0009) -[2023-10-15 15:51:59,947][52833] Updated weights for policy 0, policy_version 26670 (0.0008) -[2023-10-15 15:52:00,324][52833] Updated weights for policy 0, policy_version 26680 (0.0009) -[2023-10-15 15:52:01,053][52866] Updated weights for policy 1, policy_version 26760 (0.0008) -[2023-10-15 15:52:01,415][52866] Updated weights for policy 1, policy_version 26770 (0.0009) -[2023-10-15 15:52:01,790][52866] Updated weights for policy 1, policy_version 26780 (0.0009) -[2023-10-15 15:52:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 54755328. Throughput: 0: 1786.7, 1: 1813.3. Samples: 13692910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:52:03,442][51532] Avg episode reward: [(0, '32.900'), (1, '32.970')] -[2023-10-15 15:52:04,032][52833] Updated weights for policy 0, policy_version 26690 (0.0009) -[2023-10-15 15:52:04,400][52833] Updated weights for policy 0, policy_version 26700 (0.0008) -[2023-10-15 15:52:04,766][52833] Updated weights for policy 0, policy_version 26710 (0.0007) -[2023-10-15 15:52:05,136][52833] Updated weights for policy 0, policy_version 26720 (0.0008) -[2023-10-15 15:52:05,583][52866] Updated weights for policy 1, policy_version 26790 (0.0009) -[2023-10-15 15:52:05,958][52866] Updated weights for policy 1, policy_version 26800 (0.0007) -[2023-10-15 15:52:06,324][52866] Updated weights for policy 1, policy_version 26810 (0.0007) -[2023-10-15 15:52:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 54820864. Throughput: 0: 1790.4, 1: 1808.5. Samples: 13714386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:52:08,442][51532] Avg episode reward: [(0, '31.100'), (1, '34.380')] -[2023-10-15 15:52:08,874][52833] Updated weights for policy 0, policy_version 26730 (0.0008) -[2023-10-15 15:52:09,232][52833] Updated weights for policy 0, policy_version 26740 (0.0007) -[2023-10-15 15:52:09,600][52833] Updated weights for policy 0, policy_version 26750 (0.0007) -[2023-10-15 15:52:09,886][52866] Updated weights for policy 1, policy_version 26820 (0.0009) -[2023-10-15 15:52:10,257][52866] Updated weights for policy 1, policy_version 26830 (0.0008) -[2023-10-15 15:52:10,623][52866] Updated weights for policy 1, policy_version 26840 (0.0007) -[2023-10-15 15:52:13,175][52833] Updated weights for policy 0, policy_version 26760 (0.0008) -[2023-10-15 15:52:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 54886400. Throughput: 0: 1813.7, 1: 1810.7. Samples: 13737176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:52:13,441][51532] Avg episode reward: [(0, '33.040'), (1, '35.680')] -[2023-10-15 15:52:13,543][52833] Updated weights for policy 0, policy_version 26770 (0.0008) -[2023-10-15 15:52:13,907][52833] Updated weights for policy 0, policy_version 26780 (0.0008) -[2023-10-15 15:52:14,403][52866] Updated weights for policy 1, policy_version 26850 (0.0008) -[2023-10-15 15:52:14,773][52866] Updated weights for policy 1, policy_version 26860 (0.0009) -[2023-10-15 15:52:15,136][52866] Updated weights for policy 1, policy_version 26870 (0.0008) -[2023-10-15 15:52:15,503][52866] Updated weights for policy 1, policy_version 26880 (0.0008) -[2023-10-15 15:52:17,715][52833] Updated weights for policy 0, policy_version 26790 (0.0008) -[2023-10-15 15:52:18,091][52833] Updated weights for policy 0, policy_version 26800 (0.0008) -[2023-10-15 15:52:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 54951936. Throughput: 0: 1796.1, 1: 1811.5. Samples: 13747110. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:52:18,441][51532] Avg episode reward: [(0, '33.800'), (1, '36.700')] -[2023-10-15 15:52:18,458][52833] Updated weights for policy 0, policy_version 26810 (0.0007) -[2023-10-15 15:52:19,114][52866] Updated weights for policy 1, policy_version 26890 (0.0007) -[2023-10-15 15:52:19,480][52866] Updated weights for policy 1, policy_version 26900 (0.0009) -[2023-10-15 15:52:19,850][52866] Updated weights for policy 1, policy_version 26910 (0.0009) -[2023-10-15 15:52:22,248][52833] Updated weights for policy 0, policy_version 26820 (0.0007) -[2023-10-15 15:52:22,610][52833] Updated weights for policy 0, policy_version 26830 (0.0008) -[2023-10-15 15:52:22,975][52833] Updated weights for policy 0, policy_version 26840 (0.0008) -[2023-10-15 15:52:23,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14440.2). Total num frames: 55050240. Throughput: 0: 1808.9, 1: 1809.5. Samples: 13769250. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:52:23,441][51532] Avg episode reward: [(0, '32.190'), (1, '34.350')] -[2023-10-15 15:52:23,663][52866] Updated weights for policy 1, policy_version 26920 (0.0008) -[2023-10-15 15:52:24,025][52866] Updated weights for policy 1, policy_version 26930 (0.0010) -[2023-10-15 15:52:24,397][52866] Updated weights for policy 1, policy_version 26940 (0.0010) -[2023-10-15 15:52:26,610][52833] Updated weights for policy 0, policy_version 26850 (0.0009) -[2023-10-15 15:52:26,985][52833] Updated weights for policy 0, policy_version 26860 (0.0010) -[2023-10-15 15:52:27,348][52833] Updated weights for policy 0, policy_version 26870 (0.0009) -[2023-10-15 15:52:27,717][52833] Updated weights for policy 0, policy_version 26880 (0.0009) -[2023-10-15 15:52:28,039][52866] Updated weights for policy 1, policy_version 26950 (0.0009) -[2023-10-15 15:52:28,401][52866] Updated weights for policy 1, policy_version 26960 (0.0010) -[2023-10-15 15:52:28,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55115776. Throughput: 0: 1795.8, 1: 1814.5. Samples: 13790140. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:52:28,442][51532] Avg episode reward: [(0, '33.630'), (1, '36.350')] -[2023-10-15 15:52:28,765][52866] Updated weights for policy 1, policy_version 26970 (0.0011) -[2023-10-15 15:52:31,604][52833] Updated weights for policy 0, policy_version 26890 (0.0011) -[2023-10-15 15:52:31,965][52833] Updated weights for policy 0, policy_version 26900 (0.0008) -[2023-10-15 15:52:32,333][52833] Updated weights for policy 0, policy_version 26910 (0.0007) -[2023-10-15 15:52:32,773][52866] Updated weights for policy 1, policy_version 26980 (0.0009) -[2023-10-15 15:52:33,157][52866] Updated weights for policy 1, policy_version 26990 (0.0009) -[2023-10-15 15:52:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 55181312. Throughput: 0: 1795.5, 1: 1803.1. Samples: 13801216. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 15:52:33,442][51532] Avg episode reward: [(0, '34.870'), (1, '34.450')] -[2023-10-15 15:52:33,515][52866] Updated weights for policy 1, policy_version 27000 (0.0010) -[2023-10-15 15:52:36,128][52833] Updated weights for policy 0, policy_version 26920 (0.0008) -[2023-10-15 15:52:36,505][52833] Updated weights for policy 0, policy_version 26930 (0.0008) -[2023-10-15 15:52:36,872][52833] Updated weights for policy 0, policy_version 26940 (0.0007) -[2023-10-15 15:52:37,342][52866] Updated weights for policy 1, policy_version 27010 (0.0009) -[2023-10-15 15:52:37,704][52866] Updated weights for policy 1, policy_version 27020 (0.0009) -[2023-10-15 15:52:38,070][52866] Updated weights for policy 1, policy_version 27030 (0.0007) -[2023-10-15 15:52:38,439][52866] Updated weights for policy 1, policy_version 27040 (0.0007) -[2023-10-15 15:52:38,442][51532] Fps is (10 sec: 16382.1, 60 sec: 14745.3, 300 sec: 14440.1). Total num frames: 55279616. Throughput: 0: 1797.2, 1: 1803.4. Samples: 13822382. Policy #0 lag: (min: 5.0, avg: 7.8, max: 37.0) -[2023-10-15 15:52:38,443][51532] Avg episode reward: [(0, '35.520'), (1, '33.350')] -[2023-10-15 15:52:40,575][52833] Updated weights for policy 0, policy_version 26950 (0.0009) -[2023-10-15 15:52:40,940][52833] Updated weights for policy 0, policy_version 26960 (0.0008) -[2023-10-15 15:52:41,305][52833] Updated weights for policy 0, policy_version 26970 (0.0008) -[2023-10-15 15:52:42,283][52866] Updated weights for policy 1, policy_version 27050 (0.0010) -[2023-10-15 15:52:42,660][52866] Updated weights for policy 1, policy_version 27060 (0.0010) -[2023-10-15 15:52:43,027][52866] Updated weights for policy 1, policy_version 27070 (0.0010) -[2023-10-15 15:52:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 55345152. Throughput: 0: 1795.5, 1: 1797.2. Samples: 13843326. Policy #0 lag: (min: 5.0, avg: 7.8, max: 37.0) -[2023-10-15 15:52:43,442][51532] Avg episode reward: [(0, '36.240'), (1, '34.330')] -[2023-10-15 15:52:45,123][52833] Updated weights for policy 0, policy_version 26980 (0.0010) -[2023-10-15 15:52:45,514][52833] Updated weights for policy 0, policy_version 26990 (0.0008) -[2023-10-15 15:52:45,880][52833] Updated weights for policy 0, policy_version 27000 (0.0007) -[2023-10-15 15:52:46,775][52866] Updated weights for policy 1, policy_version 27080 (0.0007) -[2023-10-15 15:52:47,138][52866] Updated weights for policy 1, policy_version 27090 (0.0007) -[2023-10-15 15:52:47,508][52866] Updated weights for policy 1, policy_version 27100 (0.0008) -[2023-10-15 15:52:48,441][51532] Fps is (10 sec: 13108.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 55410688. Throughput: 0: 1803.6, 1: 1789.9. Samples: 13854616. Policy #0 lag: (min: 5.0, avg: 7.8, max: 37.0) -[2023-10-15 15:52:48,442][51532] Avg episode reward: [(0, '35.840'), (1, '34.970')] -[2023-10-15 15:52:49,747][52833] Updated weights for policy 0, policy_version 27010 (0.0009) -[2023-10-15 15:52:50,114][52833] Updated weights for policy 0, policy_version 27020 (0.0007) -[2023-10-15 15:52:50,490][52833] Updated weights for policy 0, policy_version 27030 (0.0011) -[2023-10-15 15:52:50,859][52833] Updated weights for policy 0, policy_version 27040 (0.0010) -[2023-10-15 15:52:51,125][52866] Updated weights for policy 1, policy_version 27110 (0.0007) -[2023-10-15 15:52:51,491][52866] Updated weights for policy 1, policy_version 27120 (0.0009) -[2023-10-15 15:52:51,848][52866] Updated weights for policy 1, policy_version 27130 (0.0008) -[2023-10-15 15:52:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 55476224. Throughput: 0: 1786.1, 1: 1793.1. Samples: 13875452. Policy #0 lag: (min: 5.0, avg: 7.8, max: 37.0) -[2023-10-15 15:52:53,442][51532] Avg episode reward: [(0, '33.730'), (1, '33.030')] -[2023-10-15 15:52:54,707][52833] Updated weights for policy 0, policy_version 27050 (0.0009) -[2023-10-15 15:52:55,073][52833] Updated weights for policy 0, policy_version 27060 (0.0009) -[2023-10-15 15:52:55,442][52833] Updated weights for policy 0, policy_version 27070 (0.0008) -[2023-10-15 15:52:55,556][52866] Updated weights for policy 1, policy_version 27140 (0.0007) -[2023-10-15 15:52:55,932][52866] Updated weights for policy 1, policy_version 27150 (0.0008) -[2023-10-15 15:52:56,287][52866] Updated weights for policy 1, policy_version 27160 (0.0009) -[2023-10-15 15:52:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 55541760. Throughput: 0: 1779.1, 1: 1784.1. Samples: 13897524. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) -[2023-10-15 15:52:58,442][51532] Avg episode reward: [(0, '35.280'), (1, '31.010')] -[2023-10-15 15:52:59,164][52833] Updated weights for policy 0, policy_version 27080 (0.0008) -[2023-10-15 15:52:59,532][52833] Updated weights for policy 0, policy_version 27090 (0.0008) -[2023-10-15 15:52:59,903][52833] Updated weights for policy 0, policy_version 27100 (0.0007) -[2023-10-15 15:53:00,030][52866] Updated weights for policy 1, policy_version 27170 (0.0010) -[2023-10-15 15:53:00,398][52866] Updated weights for policy 1, policy_version 27180 (0.0008) -[2023-10-15 15:53:00,758][52866] Updated weights for policy 1, policy_version 27190 (0.0008) -[2023-10-15 15:53:01,125][52866] Updated weights for policy 1, policy_version 27200 (0.0008) -[2023-10-15 15:53:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 55607296. Throughput: 0: 1777.7, 1: 1791.5. Samples: 13907722. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) -[2023-10-15 15:53:03,442][51532] Avg episode reward: [(0, '33.680'), (1, '29.390')] -[2023-10-15 15:53:03,465][52833] Updated weights for policy 0, policy_version 27110 (0.0009) -[2023-10-15 15:53:03,830][52833] Updated weights for policy 0, policy_version 27120 (0.0007) -[2023-10-15 15:53:04,206][52833] Updated weights for policy 0, policy_version 27130 (0.0007) -[2023-10-15 15:53:04,809][52866] Updated weights for policy 1, policy_version 27210 (0.0007) -[2023-10-15 15:53:05,178][52866] Updated weights for policy 1, policy_version 27220 (0.0007) -[2023-10-15 15:53:05,542][52866] Updated weights for policy 1, policy_version 27230 (0.0007) -[2023-10-15 15:53:07,913][52833] Updated weights for policy 0, policy_version 27140 (0.0007) -[2023-10-15 15:53:08,285][52833] Updated weights for policy 0, policy_version 27150 (0.0007) -[2023-10-15 15:53:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 55672832. Throughput: 0: 1788.5, 1: 1789.6. Samples: 13930266. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) -[2023-10-15 15:53:08,443][51532] Avg episode reward: [(0, '31.880'), (1, '31.320')] -[2023-10-15 15:53:08,659][52833] Updated weights for policy 0, policy_version 27160 (0.0008) -[2023-10-15 15:53:09,266][52866] Updated weights for policy 1, policy_version 27240 (0.0007) -[2023-10-15 15:53:09,640][52866] Updated weights for policy 1, policy_version 27250 (0.0009) -[2023-10-15 15:53:10,009][52866] Updated weights for policy 1, policy_version 27260 (0.0011) -[2023-10-15 15:53:12,324][52833] Updated weights for policy 0, policy_version 27170 (0.0008) -[2023-10-15 15:53:12,695][52833] Updated weights for policy 0, policy_version 27180 (0.0008) -[2023-10-15 15:53:13,068][52833] Updated weights for policy 0, policy_version 27190 (0.0008) -[2023-10-15 15:53:13,434][52833] Updated weights for policy 0, policy_version 27200 (0.0009) -[2023-10-15 15:53:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 55771136. Throughput: 0: 1806.5, 1: 1787.0. Samples: 13951848. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) -[2023-10-15 15:53:13,441][51532] Avg episode reward: [(0, '32.380'), (1, '30.320')] -[2023-10-15 15:53:13,822][52866] Updated weights for policy 1, policy_version 27270 (0.0008) -[2023-10-15 15:53:14,186][52866] Updated weights for policy 1, policy_version 27280 (0.0007) -[2023-10-15 15:53:14,545][52866] Updated weights for policy 1, policy_version 27290 (0.0009) -[2023-10-15 15:53:17,191][52833] Updated weights for policy 0, policy_version 27210 (0.0007) -[2023-10-15 15:53:17,555][52833] Updated weights for policy 0, policy_version 27220 (0.0008) -[2023-10-15 15:53:17,931][52833] Updated weights for policy 0, policy_version 27230 (0.0009) -[2023-10-15 15:53:18,298][52866] Updated weights for policy 1, policy_version 27300 (0.0008) -[2023-10-15 15:53:18,441][51532] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 55836672. Throughput: 0: 1796.8, 1: 1791.0. Samples: 13962668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:53:18,442][51532] Avg episode reward: [(0, '36.400'), (1, '30.050')] -[2023-10-15 15:53:18,673][52866] Updated weights for policy 1, policy_version 27310 (0.0008) -[2023-10-15 15:53:19,030][52866] Updated weights for policy 1, policy_version 27320 (0.0009) -[2023-10-15 15:53:21,657][52833] Updated weights for policy 0, policy_version 27240 (0.0008) -[2023-10-15 15:53:22,028][52833] Updated weights for policy 0, policy_version 27250 (0.0008) -[2023-10-15 15:53:22,405][52833] Updated weights for policy 0, policy_version 27260 (0.0008) -[2023-10-15 15:53:22,737][52866] Updated weights for policy 1, policy_version 27330 (0.0007) -[2023-10-15 15:53:23,102][52866] Updated weights for policy 1, policy_version 27340 (0.0007) -[2023-10-15 15:53:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 55902208. Throughput: 0: 1806.1, 1: 1801.5. Samples: 13984720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:53:23,442][51532] Avg episode reward: [(0, '33.640'), (1, '30.740')] -[2023-10-15 15:53:23,467][52866] Updated weights for policy 1, policy_version 27350 (0.0009) -[2023-10-15 15:53:23,845][52866] Updated weights for policy 1, policy_version 27360 (0.0010) -[2023-10-15 15:53:26,090][52833] Updated weights for policy 0, policy_version 27270 (0.0008) -[2023-10-15 15:53:26,468][52833] Updated weights for policy 0, policy_version 27280 (0.0008) -[2023-10-15 15:53:26,838][52833] Updated weights for policy 0, policy_version 27290 (0.0008) -[2023-10-15 15:53:27,531][52866] Updated weights for policy 1, policy_version 27370 (0.0009) -[2023-10-15 15:53:27,906][52866] Updated weights for policy 1, policy_version 27380 (0.0011) -[2023-10-15 15:53:28,272][52866] Updated weights for policy 1, policy_version 27390 (0.0010) -[2023-10-15 15:53:28,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 56000512. Throughput: 0: 1788.2, 1: 1809.9. Samples: 14005240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:53:28,441][51532] Avg episode reward: [(0, '33.460'), (1, '30.510')] -[2023-10-15 15:53:30,679][52833] Updated weights for policy 0, policy_version 27300 (0.0008) -[2023-10-15 15:53:31,068][52833] Updated weights for policy 0, policy_version 27310 (0.0010) -[2023-10-15 15:53:31,436][52833] Updated weights for policy 0, policy_version 27320 (0.0008) -[2023-10-15 15:53:32,112][52866] Updated weights for policy 1, policy_version 27400 (0.0009) -[2023-10-15 15:53:32,483][52866] Updated weights for policy 1, policy_version 27410 (0.0007) -[2023-10-15 15:53:32,847][52866] Updated weights for policy 1, policy_version 27420 (0.0007) -[2023-10-15 15:53:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 56066048. Throughput: 0: 1804.7, 1: 1803.6. Samples: 14016988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:53:33,442][51532] Avg episode reward: [(0, '32.430'), (1, '30.630')] -[2023-10-15 15:53:35,219][52833] Updated weights for policy 0, policy_version 27330 (0.0008) -[2023-10-15 15:53:35,592][52833] Updated weights for policy 0, policy_version 27340 (0.0008) -[2023-10-15 15:53:35,959][52833] Updated weights for policy 0, policy_version 27350 (0.0008) -[2023-10-15 15:53:36,323][52833] Updated weights for policy 0, policy_version 27360 (0.0009) -[2023-10-15 15:53:36,580][52866] Updated weights for policy 1, policy_version 27430 (0.0008) -[2023-10-15 15:53:36,945][52866] Updated weights for policy 1, policy_version 27440 (0.0010) -[2023-10-15 15:53:37,320][52866] Updated weights for policy 1, policy_version 27450 (0.0012) -[2023-10-15 15:53:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.8, 300 sec: 14440.1). Total num frames: 56131584. Throughput: 0: 1794.7, 1: 1814.7. Samples: 14037874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:53:38,442][51532] Avg episode reward: [(0, '34.690'), (1, '35.160')] -[2023-10-15 15:53:40,010][52833] Updated weights for policy 0, policy_version 27370 (0.0008) -[2023-10-15 15:53:40,378][52833] Updated weights for policy 0, policy_version 27380 (0.0007) -[2023-10-15 15:53:40,743][52833] Updated weights for policy 0, policy_version 27390 (0.0007) -[2023-10-15 15:53:41,059][52866] Updated weights for policy 1, policy_version 27460 (0.0009) -[2023-10-15 15:53:41,423][52866] Updated weights for policy 1, policy_version 27470 (0.0009) -[2023-10-15 15:53:41,791][52866] Updated weights for policy 1, policy_version 27480 (0.0009) -[2023-10-15 15:53:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 56197120. Throughput: 0: 1804.3, 1: 1801.3. Samples: 14059778. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) -[2023-10-15 15:53:43,442][51532] Avg episode reward: [(0, '35.650'), (1, '35.590')] -[2023-10-15 15:53:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000027392_28049408.pth... -[2023-10-15 15:53:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000027488_28147712.pth... -[2023-10-15 15:53:43,504][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000025728_26345472.pth -[2023-10-15 15:53:43,504][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000025792_26411008.pth -[2023-10-15 15:53:44,512][52833] Updated weights for policy 0, policy_version 27400 (0.0008) -[2023-10-15 15:53:44,885][52833] Updated weights for policy 0, policy_version 27410 (0.0008) -[2023-10-15 15:53:45,252][52833] Updated weights for policy 0, policy_version 27420 (0.0010) -[2023-10-15 15:53:45,628][52866] Updated weights for policy 1, policy_version 27490 (0.0010) -[2023-10-15 15:53:45,998][52866] Updated weights for policy 1, policy_version 27500 (0.0009) -[2023-10-15 15:53:46,363][52866] Updated weights for policy 1, policy_version 27510 (0.0010) -[2023-10-15 15:53:46,731][52866] Updated weights for policy 1, policy_version 27520 (0.0008) -[2023-10-15 15:53:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 56262656. Throughput: 0: 1802.8, 1: 1811.6. Samples: 14070370. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) -[2023-10-15 15:53:48,442][51532] Avg episode reward: [(0, '34.170'), (1, '37.210')] -[2023-10-15 15:53:48,929][52833] Updated weights for policy 0, policy_version 27430 (0.0008) -[2023-10-15 15:53:49,301][52833] Updated weights for policy 0, policy_version 27440 (0.0008) -[2023-10-15 15:53:49,671][52833] Updated weights for policy 0, policy_version 27450 (0.0008) -[2023-10-15 15:53:50,451][52866] Updated weights for policy 1, policy_version 27530 (0.0008) -[2023-10-15 15:53:50,818][52866] Updated weights for policy 1, policy_version 27540 (0.0009) -[2023-10-15 15:53:51,183][52866] Updated weights for policy 1, policy_version 27550 (0.0009) -[2023-10-15 15:53:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 56328192. Throughput: 0: 1792.0, 1: 1793.1. Samples: 14091592. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) -[2023-10-15 15:53:53,441][51532] Avg episode reward: [(0, '32.490'), (1, '37.010')] -[2023-10-15 15:53:53,471][52833] Updated weights for policy 0, policy_version 27460 (0.0008) -[2023-10-15 15:53:53,838][52833] Updated weights for policy 0, policy_version 27470 (0.0007) -[2023-10-15 15:53:54,208][52833] Updated weights for policy 0, policy_version 27480 (0.0008) -[2023-10-15 15:53:55,015][52866] Updated weights for policy 1, policy_version 27560 (0.0009) -[2023-10-15 15:53:55,380][52866] Updated weights for policy 1, policy_version 27570 (0.0008) -[2023-10-15 15:53:55,742][52866] Updated weights for policy 1, policy_version 27580 (0.0008) -[2023-10-15 15:53:58,024][52833] Updated weights for policy 0, policy_version 27490 (0.0008) -[2023-10-15 15:53:58,398][52833] Updated weights for policy 0, policy_version 27500 (0.0007) -[2023-10-15 15:53:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 56393728. Throughput: 0: 1809.3, 1: 1794.4. Samples: 14114016. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) -[2023-10-15 15:53:58,442][51532] Avg episode reward: [(0, '34.050'), (1, '36.750')] -[2023-10-15 15:53:58,769][52833] Updated weights for policy 0, policy_version 27510 (0.0007) -[2023-10-15 15:53:59,129][52833] Updated weights for policy 0, policy_version 27520 (0.0008) -[2023-10-15 15:53:59,617][52866] Updated weights for policy 1, policy_version 27590 (0.0008) -[2023-10-15 15:53:59,986][52866] Updated weights for policy 1, policy_version 27600 (0.0010) -[2023-10-15 15:54:00,349][52866] Updated weights for policy 1, policy_version 27610 (0.0010) -[2023-10-15 15:54:02,863][52833] Updated weights for policy 0, policy_version 27530 (0.0008) -[2023-10-15 15:54:03,239][52833] Updated weights for policy 0, policy_version 27540 (0.0008) -[2023-10-15 15:54:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 56459264. Throughput: 0: 1789.8, 1: 1795.3. Samples: 14123998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:54:03,442][51532] Avg episode reward: [(0, '35.300'), (1, '37.630')] -[2023-10-15 15:54:03,612][52833] Updated weights for policy 0, policy_version 27550 (0.0010) -[2023-10-15 15:54:04,070][52866] Updated weights for policy 1, policy_version 27620 (0.0007) -[2023-10-15 15:54:04,446][52866] Updated weights for policy 1, policy_version 27630 (0.0008) -[2023-10-15 15:54:04,816][52866] Updated weights for policy 1, policy_version 27640 (0.0008) -[2023-10-15 15:54:07,237][52833] Updated weights for policy 0, policy_version 27560 (0.0008) -[2023-10-15 15:54:07,606][52833] Updated weights for policy 0, policy_version 27570 (0.0010) -[2023-10-15 15:54:07,974][52833] Updated weights for policy 0, policy_version 27580 (0.0009) -[2023-10-15 15:54:08,399][52866] Updated weights for policy 1, policy_version 27650 (0.0010) -[2023-10-15 15:54:08,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 56557568. Throughput: 0: 1806.3, 1: 1785.3. Samples: 14146344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:54:08,442][51532] Avg episode reward: [(0, '34.040'), (1, '37.440')] -[2023-10-15 15:54:08,755][52866] Updated weights for policy 1, policy_version 27660 (0.0009) -[2023-10-15 15:54:09,134][52866] Updated weights for policy 1, policy_version 27670 (0.0009) -[2023-10-15 15:54:09,495][52866] Updated weights for policy 1, policy_version 27680 (0.0007) -[2023-10-15 15:54:11,790][52833] Updated weights for policy 0, policy_version 27590 (0.0009) -[2023-10-15 15:54:12,164][52833] Updated weights for policy 0, policy_version 27600 (0.0008) -[2023-10-15 15:54:12,527][52833] Updated weights for policy 0, policy_version 27610 (0.0011) -[2023-10-15 15:54:13,418][52866] Updated weights for policy 1, policy_version 27690 (0.0008) -[2023-10-15 15:54:13,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 56623104. Throughput: 0: 1790.7, 1: 1810.8. Samples: 14167310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:54:13,442][51532] Avg episode reward: [(0, '35.660'), (1, '37.350')] -[2023-10-15 15:54:13,786][52866] Updated weights for policy 1, policy_version 27700 (0.0009) -[2023-10-15 15:54:14,146][52866] Updated weights for policy 1, policy_version 27710 (0.0010) -[2023-10-15 15:54:16,396][52833] Updated weights for policy 0, policy_version 27620 (0.0010) -[2023-10-15 15:54:16,788][52833] Updated weights for policy 0, policy_version 27630 (0.0009) -[2023-10-15 15:54:17,165][52833] Updated weights for policy 0, policy_version 27640 (0.0009) -[2023-10-15 15:54:17,850][52866] Updated weights for policy 1, policy_version 27720 (0.0009) -[2023-10-15 15:54:18,216][52866] Updated weights for policy 1, policy_version 27730 (0.0010) -[2023-10-15 15:54:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 56688640. Throughput: 0: 1801.1, 1: 1788.7. Samples: 14178530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:54:18,442][51532] Avg episode reward: [(0, '35.560'), (1, '37.480')] -[2023-10-15 15:54:18,582][52866] Updated weights for policy 1, policy_version 27740 (0.0011) -[2023-10-15 15:54:20,662][52833] Updated weights for policy 0, policy_version 27650 (0.0008) -[2023-10-15 15:54:21,025][52833] Updated weights for policy 0, policy_version 27660 (0.0008) -[2023-10-15 15:54:21,398][52833] Updated weights for policy 0, policy_version 27670 (0.0008) -[2023-10-15 15:54:21,769][52833] Updated weights for policy 0, policy_version 27680 (0.0008) -[2023-10-15 15:54:22,363][52866] Updated weights for policy 1, policy_version 27750 (0.0009) -[2023-10-15 15:54:22,730][52866] Updated weights for policy 1, policy_version 27760 (0.0007) -[2023-10-15 15:54:23,095][52866] Updated weights for policy 1, policy_version 27770 (0.0007) -[2023-10-15 15:54:23,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 56786944. Throughput: 0: 1793.7, 1: 1799.8. Samples: 14199582. Policy #0 lag: (min: 20.0, avg: 20.9, max: 40.0) -[2023-10-15 15:54:23,442][51532] Avg episode reward: [(0, '37.300'), (1, '38.280')] -[2023-10-15 15:54:25,552][52833] Updated weights for policy 0, policy_version 27690 (0.0008) -[2023-10-15 15:54:25,919][52833] Updated weights for policy 0, policy_version 27700 (0.0009) -[2023-10-15 15:54:26,281][52833] Updated weights for policy 0, policy_version 27710 (0.0011) -[2023-10-15 15:54:26,807][52866] Updated weights for policy 1, policy_version 27780 (0.0009) -[2023-10-15 15:54:27,181][52866] Updated weights for policy 1, policy_version 27790 (0.0010) -[2023-10-15 15:54:27,555][52866] Updated weights for policy 1, policy_version 27800 (0.0008) -[2023-10-15 15:54:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56852480. Throughput: 0: 1784.6, 1: 1786.7. Samples: 14220484. Policy #0 lag: (min: 20.0, avg: 20.9, max: 40.0) -[2023-10-15 15:54:28,442][51532] Avg episode reward: [(0, '37.610'), (1, '38.670')] -[2023-10-15 15:54:28,454][52410] Saving new best policy, reward=37.610! -[2023-10-15 15:54:29,953][52833] Updated weights for policy 0, policy_version 27720 (0.0009) -[2023-10-15 15:54:30,325][52833] Updated weights for policy 0, policy_version 27730 (0.0008) -[2023-10-15 15:54:30,704][52833] Updated weights for policy 0, policy_version 27740 (0.0008) -[2023-10-15 15:54:31,231][52866] Updated weights for policy 1, policy_version 27810 (0.0008) -[2023-10-15 15:54:31,592][52866] Updated weights for policy 1, policy_version 27820 (0.0010) -[2023-10-15 15:54:31,963][52866] Updated weights for policy 1, policy_version 27830 (0.0007) -[2023-10-15 15:54:32,333][52866] Updated weights for policy 1, policy_version 27840 (0.0009) -[2023-10-15 15:54:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56918016. Throughput: 0: 1789.4, 1: 1808.9. Samples: 14232292. Policy #0 lag: (min: 20.0, avg: 20.9, max: 40.0) -[2023-10-15 15:54:33,442][51532] Avg episode reward: [(0, '36.830'), (1, '37.800')] -[2023-10-15 15:54:34,440][52833] Updated weights for policy 0, policy_version 27750 (0.0009) -[2023-10-15 15:54:34,805][52833] Updated weights for policy 0, policy_version 27760 (0.0010) -[2023-10-15 15:54:35,170][52833] Updated weights for policy 0, policy_version 27770 (0.0008) -[2023-10-15 15:54:36,083][52866] Updated weights for policy 1, policy_version 27850 (0.0009) -[2023-10-15 15:54:36,444][52866] Updated weights for policy 1, policy_version 27860 (0.0009) -[2023-10-15 15:54:36,811][52866] Updated weights for policy 1, policy_version 27870 (0.0009) -[2023-10-15 15:54:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56983552. Throughput: 0: 1798.0, 1: 1796.0. Samples: 14253322. Policy #0 lag: (min: 20.0, avg: 20.9, max: 40.0) -[2023-10-15 15:54:38,442][51532] Avg episode reward: [(0, '37.150'), (1, '36.920')] -[2023-10-15 15:54:39,064][52833] Updated weights for policy 0, policy_version 27780 (0.0007) -[2023-10-15 15:54:39,428][52833] Updated weights for policy 0, policy_version 27790 (0.0009) -[2023-10-15 15:54:39,796][52833] Updated weights for policy 0, policy_version 27800 (0.0009) -[2023-10-15 15:54:40,513][52866] Updated weights for policy 1, policy_version 27880 (0.0010) -[2023-10-15 15:54:40,895][52866] Updated weights for policy 1, policy_version 27890 (0.0011) -[2023-10-15 15:54:41,271][52866] Updated weights for policy 1, policy_version 27900 (0.0010) -[2023-10-15 15:54:43,362][52833] Updated weights for policy 0, policy_version 27810 (0.0009) -[2023-10-15 15:54:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 57049088. Throughput: 0: 1795.3, 1: 1802.8. Samples: 14275930. Policy #0 lag: (min: 20.0, avg: 20.9, max: 40.0) -[2023-10-15 15:54:43,442][51532] Avg episode reward: [(0, '38.270'), (1, '35.530')] -[2023-10-15 15:54:43,732][52833] Updated weights for policy 0, policy_version 27820 (0.0009) -[2023-10-15 15:54:44,093][52833] Updated weights for policy 0, policy_version 27830 (0.0009) -[2023-10-15 15:54:44,460][52410] Saving new best policy, reward=38.270! -[2023-10-15 15:54:44,460][52833] Updated weights for policy 0, policy_version 27840 (0.0008) -[2023-10-15 15:54:45,002][52866] Updated weights for policy 1, policy_version 27910 (0.0011) -[2023-10-15 15:54:45,376][52866] Updated weights for policy 1, policy_version 27920 (0.0009) -[2023-10-15 15:54:45,753][52866] Updated weights for policy 1, policy_version 27930 (0.0009) -[2023-10-15 15:54:48,159][52833] Updated weights for policy 0, policy_version 27850 (0.0009) -[2023-10-15 15:54:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 57114624. Throughput: 0: 1798.8, 1: 1799.3. Samples: 14285910. Policy #0 lag: (min: 4.0, avg: 4.3, max: 16.0) -[2023-10-15 15:54:48,441][51532] Avg episode reward: [(0, '37.680'), (1, '37.950')] -[2023-10-15 15:54:48,522][52833] Updated weights for policy 0, policy_version 27860 (0.0009) -[2023-10-15 15:54:48,893][52833] Updated weights for policy 0, policy_version 27870 (0.0007) -[2023-10-15 15:54:49,472][52866] Updated weights for policy 1, policy_version 27940 (0.0009) -[2023-10-15 15:54:49,834][52866] Updated weights for policy 1, policy_version 27950 (0.0009) -[2023-10-15 15:54:50,200][52866] Updated weights for policy 1, policy_version 27960 (0.0011) -[2023-10-15 15:54:52,635][52833] Updated weights for policy 0, policy_version 27880 (0.0008) -[2023-10-15 15:54:52,998][52833] Updated weights for policy 0, policy_version 27890 (0.0008) -[2023-10-15 15:54:53,374][52833] Updated weights for policy 0, policy_version 27900 (0.0009) -[2023-10-15 15:54:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 57180160. Throughput: 0: 1800.4, 1: 1799.4. Samples: 14308338. Policy #0 lag: (min: 4.0, avg: 4.3, max: 16.0) -[2023-10-15 15:54:53,442][51532] Avg episode reward: [(0, '35.420'), (1, '37.500')] -[2023-10-15 15:54:53,919][52866] Updated weights for policy 1, policy_version 27970 (0.0008) -[2023-10-15 15:54:54,291][52866] Updated weights for policy 1, policy_version 27980 (0.0008) -[2023-10-15 15:54:54,651][52866] Updated weights for policy 1, policy_version 27990 (0.0008) -[2023-10-15 15:54:55,018][52866] Updated weights for policy 1, policy_version 28000 (0.0011) -[2023-10-15 15:54:57,260][52833] Updated weights for policy 0, policy_version 27910 (0.0009) -[2023-10-15 15:54:57,636][52833] Updated weights for policy 0, policy_version 27920 (0.0010) -[2023-10-15 15:54:58,001][52833] Updated weights for policy 0, policy_version 27930 (0.0011) -[2023-10-15 15:54:58,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 57278464. Throughput: 0: 1809.9, 1: 1799.1. Samples: 14329712. Policy #0 lag: (min: 4.0, avg: 4.3, max: 16.0) -[2023-10-15 15:54:58,442][51532] Avg episode reward: [(0, '34.630'), (1, '37.220')] -[2023-10-15 15:54:58,895][52866] Updated weights for policy 1, policy_version 28010 (0.0009) -[2023-10-15 15:54:59,264][52866] Updated weights for policy 1, policy_version 28020 (0.0009) -[2023-10-15 15:54:59,636][52866] Updated weights for policy 1, policy_version 28030 (0.0008) -[2023-10-15 15:55:01,836][52833] Updated weights for policy 0, policy_version 27940 (0.0008) -[2023-10-15 15:55:02,210][52833] Updated weights for policy 0, policy_version 27950 (0.0008) -[2023-10-15 15:55:02,582][52833] Updated weights for policy 0, policy_version 27960 (0.0009) -[2023-10-15 15:55:03,401][52866] Updated weights for policy 1, policy_version 28040 (0.0008) -[2023-10-15 15:55:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 57344000. Throughput: 0: 1798.4, 1: 1797.8. Samples: 14340360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:03,442][51532] Avg episode reward: [(0, '34.950'), (1, '35.010')] -[2023-10-15 15:55:03,774][52866] Updated weights for policy 1, policy_version 28050 (0.0009) -[2023-10-15 15:55:04,136][52866] Updated weights for policy 1, policy_version 28060 (0.0010) -[2023-10-15 15:55:06,440][52833] Updated weights for policy 0, policy_version 27970 (0.0009) -[2023-10-15 15:55:06,807][52833] Updated weights for policy 0, policy_version 27980 (0.0007) -[2023-10-15 15:55:07,182][52833] Updated weights for policy 0, policy_version 27990 (0.0007) -[2023-10-15 15:55:07,543][52833] Updated weights for policy 0, policy_version 28000 (0.0008) -[2023-10-15 15:55:07,889][52866] Updated weights for policy 1, policy_version 28070 (0.0012) -[2023-10-15 15:55:08,264][52866] Updated weights for policy 1, policy_version 28080 (0.0011) -[2023-10-15 15:55:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 57409536. Throughput: 0: 1815.1, 1: 1800.4. Samples: 14362276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:08,441][51532] Avg episode reward: [(0, '35.080'), (1, '32.690')] -[2023-10-15 15:55:08,621][52866] Updated weights for policy 1, policy_version 28090 (0.0008) -[2023-10-15 15:55:11,294][52833] Updated weights for policy 0, policy_version 28010 (0.0007) -[2023-10-15 15:55:11,666][52833] Updated weights for policy 0, policy_version 28020 (0.0010) -[2023-10-15 15:55:12,043][52833] Updated weights for policy 0, policy_version 28030 (0.0009) -[2023-10-15 15:55:12,211][52866] Updated weights for policy 1, policy_version 28100 (0.0009) -[2023-10-15 15:55:12,582][52866] Updated weights for policy 1, policy_version 28110 (0.0008) -[2023-10-15 15:55:12,947][52866] Updated weights for policy 1, policy_version 28120 (0.0008) -[2023-10-15 15:55:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 57507840. Throughput: 0: 1800.4, 1: 1810.1. Samples: 14382958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:13,442][51532] Avg episode reward: [(0, '35.440'), (1, '34.170')] -[2023-10-15 15:55:15,677][52833] Updated weights for policy 0, policy_version 28040 (0.0007) -[2023-10-15 15:55:16,051][52833] Updated weights for policy 0, policy_version 28050 (0.0007) -[2023-10-15 15:55:16,419][52833] Updated weights for policy 0, policy_version 28060 (0.0009) -[2023-10-15 15:55:16,579][52866] Updated weights for policy 1, policy_version 28130 (0.0008) -[2023-10-15 15:55:16,942][52866] Updated weights for policy 1, policy_version 28140 (0.0008) -[2023-10-15 15:55:17,316][52866] Updated weights for policy 1, policy_version 28150 (0.0008) -[2023-10-15 15:55:17,687][52866] Updated weights for policy 1, policy_version 28160 (0.0009) -[2023-10-15 15:55:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 57573376. Throughput: 0: 1816.0, 1: 1796.3. Samples: 14394844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:18,441][51532] Avg episode reward: [(0, '33.490'), (1, '33.370')] -[2023-10-15 15:55:20,201][52833] Updated weights for policy 0, policy_version 28070 (0.0010) -[2023-10-15 15:55:20,574][52833] Updated weights for policy 0, policy_version 28080 (0.0008) -[2023-10-15 15:55:20,941][52833] Updated weights for policy 0, policy_version 28090 (0.0009) -[2023-10-15 15:55:21,365][52866] Updated weights for policy 1, policy_version 28170 (0.0007) -[2023-10-15 15:55:21,733][52866] Updated weights for policy 1, policy_version 28180 (0.0007) -[2023-10-15 15:55:22,094][52866] Updated weights for policy 1, policy_version 28190 (0.0008) -[2023-10-15 15:55:23,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 57638912. Throughput: 0: 1791.9, 1: 1810.1. Samples: 14415414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:23,441][51532] Avg episode reward: [(0, '34.210'), (1, '33.800')] -[2023-10-15 15:55:24,803][52833] Updated weights for policy 0, policy_version 28100 (0.0009) -[2023-10-15 15:55:25,175][52833] Updated weights for policy 0, policy_version 28110 (0.0008) -[2023-10-15 15:55:25,553][52833] Updated weights for policy 0, policy_version 28120 (0.0008) -[2023-10-15 15:55:25,851][52866] Updated weights for policy 1, policy_version 28200 (0.0008) -[2023-10-15 15:55:26,213][52866] Updated weights for policy 1, policy_version 28210 (0.0010) -[2023-10-15 15:55:26,591][52866] Updated weights for policy 1, policy_version 28220 (0.0012) -[2023-10-15 15:55:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57704448. Throughput: 0: 1788.4, 1: 1797.1. Samples: 14437278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:28,441][51532] Avg episode reward: [(0, '34.900'), (1, '34.470')] -[2023-10-15 15:55:29,318][52833] Updated weights for policy 0, policy_version 28130 (0.0009) -[2023-10-15 15:55:29,681][52833] Updated weights for policy 0, policy_version 28140 (0.0009) -[2023-10-15 15:55:30,042][52833] Updated weights for policy 0, policy_version 28150 (0.0007) -[2023-10-15 15:55:30,240][52866] Updated weights for policy 1, policy_version 28230 (0.0008) -[2023-10-15 15:55:30,408][52833] Updated weights for policy 0, policy_version 28160 (0.0008) -[2023-10-15 15:55:30,602][52866] Updated weights for policy 1, policy_version 28240 (0.0008) -[2023-10-15 15:55:30,970][52866] Updated weights for policy 1, policy_version 28250 (0.0011) -[2023-10-15 15:55:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 57769984. Throughput: 0: 1779.6, 1: 1810.6. Samples: 14447470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:33,441][51532] Avg episode reward: [(0, '35.240'), (1, '35.320')] -[2023-10-15 15:55:34,044][52833] Updated weights for policy 0, policy_version 28170 (0.0008) -[2023-10-15 15:55:34,403][52833] Updated weights for policy 0, policy_version 28180 (0.0007) -[2023-10-15 15:55:34,661][52866] Updated weights for policy 1, policy_version 28260 (0.0008) -[2023-10-15 15:55:34,768][52833] Updated weights for policy 0, policy_version 28190 (0.0009) -[2023-10-15 15:55:35,031][52866] Updated weights for policy 1, policy_version 28270 (0.0007) -[2023-10-15 15:55:35,395][52866] Updated weights for policy 1, policy_version 28280 (0.0007) -[2023-10-15 15:55:38,441][51532] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 57835520. Throughput: 0: 1778.0, 1: 1807.1. Samples: 14469666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:38,442][51532] Avg episode reward: [(0, '34.120'), (1, '31.840')] -[2023-10-15 15:55:38,735][52833] Updated weights for policy 0, policy_version 28200 (0.0009) -[2023-10-15 15:55:39,096][52866] Updated weights for policy 1, policy_version 28290 (0.0007) -[2023-10-15 15:55:39,103][52833] Updated weights for policy 0, policy_version 28210 (0.0010) -[2023-10-15 15:55:39,472][52866] Updated weights for policy 1, policy_version 28300 (0.0007) -[2023-10-15 15:55:39,475][52833] Updated weights for policy 0, policy_version 28220 (0.0009) -[2023-10-15 15:55:39,836][52866] Updated weights for policy 1, policy_version 28310 (0.0008) -[2023-10-15 15:55:40,204][52866] Updated weights for policy 1, policy_version 28320 (0.0008) -[2023-10-15 15:55:43,287][52833] Updated weights for policy 0, policy_version 28230 (0.0008) -[2023-10-15 15:55:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 57901056. Throughput: 0: 1800.7, 1: 1804.4. Samples: 14491944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:43,442][51532] Avg episode reward: [(0, '35.460'), (1, '31.550')] -[2023-10-15 15:55:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000028320_28999680.pth... -[2023-10-15 15:55:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000026656_27295744.pth -[2023-10-15 15:55:43,657][52833] Updated weights for policy 0, policy_version 28240 (0.0008) -[2023-10-15 15:55:44,019][52833] Updated weights for policy 0, policy_version 28250 (0.0009) -[2023-10-15 15:55:44,200][52866] Updated weights for policy 1, policy_version 28330 (0.0008) -[2023-10-15 15:55:44,242][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000028256_28934144.pth... -[2023-10-15 15:55:44,270][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000026560_27197440.pth -[2023-10-15 15:55:44,571][52866] Updated weights for policy 1, policy_version 28340 (0.0007) -[2023-10-15 15:55:44,935][52866] Updated weights for policy 1, policy_version 28350 (0.0008) -[2023-10-15 15:55:47,745][52833] Updated weights for policy 0, policy_version 28260 (0.0008) -[2023-10-15 15:55:48,145][52833] Updated weights for policy 0, policy_version 28270 (0.0007) -[2023-10-15 15:55:48,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 57966592. Throughput: 0: 1780.4, 1: 1799.8. Samples: 14501470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:48,442][51532] Avg episode reward: [(0, '34.910'), (1, '29.990')] -[2023-10-15 15:55:48,514][52833] Updated weights for policy 0, policy_version 28280 (0.0010) -[2023-10-15 15:55:48,750][52866] Updated weights for policy 1, policy_version 28360 (0.0010) -[2023-10-15 15:55:49,124][52866] Updated weights for policy 1, policy_version 28370 (0.0008) -[2023-10-15 15:55:49,502][52866] Updated weights for policy 1, policy_version 28380 (0.0008) -[2023-10-15 15:55:52,244][52833] Updated weights for policy 0, policy_version 28290 (0.0007) -[2023-10-15 15:55:52,604][52833] Updated weights for policy 0, policy_version 28300 (0.0007) -[2023-10-15 15:55:52,981][52833] Updated weights for policy 0, policy_version 28310 (0.0009) -[2023-10-15 15:55:53,213][52866] Updated weights for policy 1, policy_version 28390 (0.0008) -[2023-10-15 15:55:53,348][52833] Updated weights for policy 0, policy_version 28320 (0.0008) -[2023-10-15 15:55:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 58064896. Throughput: 0: 1792.6, 1: 1795.0. Samples: 14523716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:53,442][51532] Avg episode reward: [(0, '35.380'), (1, '30.930')] -[2023-10-15 15:55:53,574][52866] Updated weights for policy 1, policy_version 28400 (0.0007) -[2023-10-15 15:55:53,947][52866] Updated weights for policy 1, policy_version 28410 (0.0008) -[2023-10-15 15:55:56,950][52833] Updated weights for policy 0, policy_version 28330 (0.0007) -[2023-10-15 15:55:57,316][52833] Updated weights for policy 0, policy_version 28340 (0.0007) -[2023-10-15 15:55:57,669][52833] Updated weights for policy 0, policy_version 28350 (0.0007) -[2023-10-15 15:55:57,867][52866] Updated weights for policy 1, policy_version 28420 (0.0008) -[2023-10-15 15:55:58,224][52866] Updated weights for policy 1, policy_version 28430 (0.0008) -[2023-10-15 15:55:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 58130432. Throughput: 0: 1781.9, 1: 1803.1. Samples: 14544284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:55:58,442][51532] Avg episode reward: [(0, '34.130'), (1, '27.560')] -[2023-10-15 15:55:58,592][52866] Updated weights for policy 1, policy_version 28440 (0.0008) -[2023-10-15 15:56:01,487][52833] Updated weights for policy 0, policy_version 28360 (0.0007) -[2023-10-15 15:56:01,861][52833] Updated weights for policy 0, policy_version 28370 (0.0009) -[2023-10-15 15:56:02,229][52833] Updated weights for policy 0, policy_version 28380 (0.0008) -[2023-10-15 15:56:02,306][52866] Updated weights for policy 1, policy_version 28450 (0.0010) -[2023-10-15 15:56:02,674][52866] Updated weights for policy 1, policy_version 28460 (0.0007) -[2023-10-15 15:56:03,041][52866] Updated weights for policy 1, policy_version 28470 (0.0007) -[2023-10-15 15:56:03,411][52866] Updated weights for policy 1, policy_version 28480 (0.0007) -[2023-10-15 15:56:03,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 58228736. Throughput: 0: 1794.7, 1: 1782.5. Samples: 14555820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:56:03,442][51532] Avg episode reward: [(0, '34.860'), (1, '28.200')] -[2023-10-15 15:56:05,828][52833] Updated weights for policy 0, policy_version 28390 (0.0009) -[2023-10-15 15:56:06,193][52833] Updated weights for policy 0, policy_version 28400 (0.0009) -[2023-10-15 15:56:06,563][52833] Updated weights for policy 0, policy_version 28410 (0.0009) -[2023-10-15 15:56:07,109][52866] Updated weights for policy 1, policy_version 28490 (0.0011) -[2023-10-15 15:56:07,477][52866] Updated weights for policy 1, policy_version 28500 (0.0009) -[2023-10-15 15:56:07,834][52866] Updated weights for policy 1, policy_version 28510 (0.0009) -[2023-10-15 15:56:08,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 58294272. Throughput: 0: 1781.9, 1: 1799.8. Samples: 14576590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:56:08,442][51532] Avg episode reward: [(0, '36.260'), (1, '28.130')] -[2023-10-15 15:56:10,414][52833] Updated weights for policy 0, policy_version 28420 (0.0010) -[2023-10-15 15:56:10,784][52833] Updated weights for policy 0, policy_version 28430 (0.0010) -[2023-10-15 15:56:11,159][52833] Updated weights for policy 0, policy_version 28440 (0.0010) -[2023-10-15 15:56:11,609][52866] Updated weights for policy 1, policy_version 28520 (0.0010) -[2023-10-15 15:56:11,976][52866] Updated weights for policy 1, policy_version 28530 (0.0011) -[2023-10-15 15:56:12,338][52866] Updated weights for policy 1, policy_version 28540 (0.0007) -[2023-10-15 15:56:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58359808. Throughput: 0: 1788.4, 1: 1784.6. Samples: 14598062. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-15 15:56:13,442][51532] Avg episode reward: [(0, '35.630'), (1, '30.420')] -[2023-10-15 15:56:14,973][52833] Updated weights for policy 0, policy_version 28450 (0.0008) -[2023-10-15 15:56:15,338][52833] Updated weights for policy 0, policy_version 28460 (0.0008) -[2023-10-15 15:56:15,718][52833] Updated weights for policy 0, policy_version 28470 (0.0010) -[2023-10-15 15:56:16,082][52833] Updated weights for policy 0, policy_version 28480 (0.0009) -[2023-10-15 15:56:16,201][52866] Updated weights for policy 1, policy_version 28550 (0.0007) -[2023-10-15 15:56:16,569][52866] Updated weights for policy 1, policy_version 28560 (0.0008) -[2023-10-15 15:56:16,937][52866] Updated weights for policy 1, policy_version 28570 (0.0008) -[2023-10-15 15:56:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 58425344. Throughput: 0: 1797.9, 1: 1807.7. Samples: 14609720. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-15 15:56:18,442][51532] Avg episode reward: [(0, '34.760'), (1, '30.190')] -[2023-10-15 15:56:19,674][52833] Updated weights for policy 0, policy_version 28490 (0.0010) -[2023-10-15 15:56:20,047][52833] Updated weights for policy 0, policy_version 28500 (0.0007) -[2023-10-15 15:56:20,428][52833] Updated weights for policy 0, policy_version 28510 (0.0008) -[2023-10-15 15:56:20,666][52866] Updated weights for policy 1, policy_version 28580 (0.0007) -[2023-10-15 15:56:21,031][52866] Updated weights for policy 1, policy_version 28590 (0.0008) -[2023-10-15 15:56:21,400][52866] Updated weights for policy 1, policy_version 28600 (0.0012) -[2023-10-15 15:56:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 58490880. Throughput: 0: 1792.2, 1: 1781.4. Samples: 14630476. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-15 15:56:23,441][51532] Avg episode reward: [(0, '33.410'), (1, '33.820')] -[2023-10-15 15:56:24,269][52833] Updated weights for policy 0, policy_version 28520 (0.0007) -[2023-10-15 15:56:24,640][52833] Updated weights for policy 0, policy_version 28530 (0.0009) -[2023-10-15 15:56:25,013][52833] Updated weights for policy 0, policy_version 28540 (0.0009) -[2023-10-15 15:56:25,098][52866] Updated weights for policy 1, policy_version 28610 (0.0009) -[2023-10-15 15:56:25,467][52866] Updated weights for policy 1, policy_version 28620 (0.0009) -[2023-10-15 15:56:25,829][52866] Updated weights for policy 1, policy_version 28630 (0.0007) -[2023-10-15 15:56:26,198][52866] Updated weights for policy 1, policy_version 28640 (0.0008) -[2023-10-15 15:56:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 58556416. Throughput: 0: 1793.6, 1: 1786.3. Samples: 14653040. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-15 15:56:28,441][51532] Avg episode reward: [(0, '35.430'), (1, '34.910')] -[2023-10-15 15:56:28,752][52833] Updated weights for policy 0, policy_version 28550 (0.0008) -[2023-10-15 15:56:29,114][52833] Updated weights for policy 0, policy_version 28560 (0.0008) -[2023-10-15 15:56:29,479][52833] Updated weights for policy 0, policy_version 28570 (0.0009) -[2023-10-15 15:56:29,822][52866] Updated weights for policy 1, policy_version 28650 (0.0007) -[2023-10-15 15:56:30,195][52866] Updated weights for policy 1, policy_version 28660 (0.0007) -[2023-10-15 15:56:30,571][52866] Updated weights for policy 1, policy_version 28670 (0.0008) -[2023-10-15 15:56:33,289][52833] Updated weights for policy 0, policy_version 28580 (0.0008) -[2023-10-15 15:56:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 58621952. Throughput: 0: 1792.3, 1: 1793.1. Samples: 14662810. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-15 15:56:33,441][51532] Avg episode reward: [(0, '34.960'), (1, '34.330')] -[2023-10-15 15:56:33,682][52833] Updated weights for policy 0, policy_version 28590 (0.0008) -[2023-10-15 15:56:34,051][52833] Updated weights for policy 0, policy_version 28600 (0.0009) -[2023-10-15 15:56:34,353][52866] Updated weights for policy 1, policy_version 28680 (0.0008) -[2023-10-15 15:56:34,715][52866] Updated weights for policy 1, policy_version 28690 (0.0008) -[2023-10-15 15:56:35,080][52866] Updated weights for policy 1, policy_version 28700 (0.0008) -[2023-10-15 15:56:37,740][52833] Updated weights for policy 0, policy_version 28610 (0.0010) -[2023-10-15 15:56:38,113][52833] Updated weights for policy 0, policy_version 28620 (0.0008) -[2023-10-15 15:56:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 58687488. Throughput: 0: 1793.6, 1: 1793.6. Samples: 14685136. Policy #0 lag: (min: 17.0, avg: 30.4, max: 49.0) -[2023-10-15 15:56:38,442][51532] Avg episode reward: [(0, '36.530'), (1, '34.920')] -[2023-10-15 15:56:38,483][52833] Updated weights for policy 0, policy_version 28630 (0.0008) -[2023-10-15 15:56:38,800][52866] Updated weights for policy 1, policy_version 28710 (0.0008) -[2023-10-15 15:56:38,850][52833] Updated weights for policy 0, policy_version 28640 (0.0009) -[2023-10-15 15:56:39,167][52866] Updated weights for policy 1, policy_version 28720 (0.0008) -[2023-10-15 15:56:39,526][52866] Updated weights for policy 1, policy_version 28730 (0.0008) -[2023-10-15 15:56:42,488][52833] Updated weights for policy 0, policy_version 28650 (0.0008) -[2023-10-15 15:56:42,852][52833] Updated weights for policy 0, policy_version 28660 (0.0008) -[2023-10-15 15:56:43,228][52833] Updated weights for policy 0, policy_version 28670 (0.0008) -[2023-10-15 15:56:43,258][52866] Updated weights for policy 1, policy_version 28740 (0.0008) -[2023-10-15 15:56:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 58785792. Throughput: 0: 1799.5, 1: 1807.3. Samples: 14706590. Policy #0 lag: (min: 17.0, avg: 30.4, max: 49.0) -[2023-10-15 15:56:43,442][51532] Avg episode reward: [(0, '36.050'), (1, '34.050')] -[2023-10-15 15:56:43,632][52866] Updated weights for policy 1, policy_version 28750 (0.0007) -[2023-10-15 15:56:43,991][52866] Updated weights for policy 1, policy_version 28760 (0.0008) -[2023-10-15 15:56:47,026][52833] Updated weights for policy 0, policy_version 28680 (0.0007) -[2023-10-15 15:56:47,393][52833] Updated weights for policy 0, policy_version 28690 (0.0007) -[2023-10-15 15:56:47,767][52833] Updated weights for policy 0, policy_version 28700 (0.0008) -[2023-10-15 15:56:47,803][52866] Updated weights for policy 1, policy_version 28770 (0.0008) -[2023-10-15 15:56:48,167][52866] Updated weights for policy 1, policy_version 28780 (0.0009) -[2023-10-15 15:56:48,442][51532] Fps is (10 sec: 16381.6, 60 sec: 14745.2, 300 sec: 14329.0). Total num frames: 58851328. Throughput: 0: 1786.8, 1: 1804.2. Samples: 14717422. Policy #0 lag: (min: 17.0, avg: 30.4, max: 49.0) -[2023-10-15 15:56:48,443][51532] Avg episode reward: [(0, '36.590'), (1, '34.240')] -[2023-10-15 15:56:48,542][52866] Updated weights for policy 1, policy_version 28790 (0.0009) -[2023-10-15 15:56:48,911][52866] Updated weights for policy 1, policy_version 28800 (0.0008) -[2023-10-15 15:56:51,463][52833] Updated weights for policy 0, policy_version 28710 (0.0009) -[2023-10-15 15:56:51,827][52833] Updated weights for policy 0, policy_version 28720 (0.0009) -[2023-10-15 15:56:52,202][52833] Updated weights for policy 0, policy_version 28730 (0.0009) -[2023-10-15 15:56:52,761][52866] Updated weights for policy 1, policy_version 28810 (0.0010) -[2023-10-15 15:56:53,123][52866] Updated weights for policy 1, policy_version 28820 (0.0011) -[2023-10-15 15:56:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 58916864. Throughput: 0: 1803.6, 1: 1804.8. Samples: 14738968. Policy #0 lag: (min: 17.0, avg: 30.4, max: 49.0) -[2023-10-15 15:56:53,441][51532] Avg episode reward: [(0, '38.430'), (1, '32.580')] -[2023-10-15 15:56:53,442][52410] Saving new best policy, reward=38.430! -[2023-10-15 15:56:53,487][52866] Updated weights for policy 1, policy_version 28830 (0.0007) -[2023-10-15 15:56:55,921][52833] Updated weights for policy 0, policy_version 28740 (0.0009) -[2023-10-15 15:56:56,290][52833] Updated weights for policy 0, policy_version 28750 (0.0008) -[2023-10-15 15:56:56,659][52833] Updated weights for policy 0, policy_version 28760 (0.0007) -[2023-10-15 15:56:57,076][52866] Updated weights for policy 1, policy_version 28840 (0.0008) -[2023-10-15 15:56:57,445][52866] Updated weights for policy 1, policy_version 28850 (0.0008) -[2023-10-15 15:56:57,816][52866] Updated weights for policy 1, policy_version 28860 (0.0008) -[2023-10-15 15:56:58,441][51532] Fps is (10 sec: 16386.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 59015168. Throughput: 0: 1783.6, 1: 1797.0. Samples: 14759192. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 15:56:58,442][51532] Avg episode reward: [(0, '38.970'), (1, '33.320')] -[2023-10-15 15:56:58,453][52410] Saving new best policy, reward=38.970! -[2023-10-15 15:57:00,609][52833] Updated weights for policy 0, policy_version 28770 (0.0008) -[2023-10-15 15:57:00,987][52833] Updated weights for policy 0, policy_version 28780 (0.0008) -[2023-10-15 15:57:01,349][52833] Updated weights for policy 0, policy_version 28790 (0.0009) -[2023-10-15 15:57:01,669][52866] Updated weights for policy 1, policy_version 28870 (0.0007) -[2023-10-15 15:57:01,719][52833] Updated weights for policy 0, policy_version 28800 (0.0009) -[2023-10-15 15:57:02,030][52866] Updated weights for policy 1, policy_version 28880 (0.0009) -[2023-10-15 15:57:02,394][52866] Updated weights for policy 1, policy_version 28890 (0.0009) -[2023-10-15 15:57:03,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 59080704. Throughput: 0: 1798.5, 1: 1788.7. Samples: 14771146. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 15:57:03,442][51532] Avg episode reward: [(0, '37.710'), (1, '32.960')] -[2023-10-15 15:57:05,526][52833] Updated weights for policy 0, policy_version 28810 (0.0008) -[2023-10-15 15:57:05,906][52833] Updated weights for policy 0, policy_version 28820 (0.0008) -[2023-10-15 15:57:06,078][52866] Updated weights for policy 1, policy_version 28900 (0.0008) -[2023-10-15 15:57:06,272][52833] Updated weights for policy 0, policy_version 28830 (0.0009) -[2023-10-15 15:57:06,442][52866] Updated weights for policy 1, policy_version 28910 (0.0009) -[2023-10-15 15:57:06,813][52866] Updated weights for policy 1, policy_version 28920 (0.0011) -[2023-10-15 15:57:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59146240. Throughput: 0: 1776.7, 1: 1796.8. Samples: 14791282. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 15:57:08,442][51532] Avg episode reward: [(0, '35.750'), (1, '35.380')] -[2023-10-15 15:57:10,071][52833] Updated weights for policy 0, policy_version 28840 (0.0008) -[2023-10-15 15:57:10,405][52866] Updated weights for policy 1, policy_version 28930 (0.0010) -[2023-10-15 15:57:10,438][52833] Updated weights for policy 0, policy_version 28850 (0.0007) -[2023-10-15 15:57:10,778][52866] Updated weights for policy 1, policy_version 28940 (0.0007) -[2023-10-15 15:57:10,813][52833] Updated weights for policy 0, policy_version 28860 (0.0007) -[2023-10-15 15:57:11,148][52866] Updated weights for policy 1, policy_version 28950 (0.0007) -[2023-10-15 15:57:11,511][52866] Updated weights for policy 1, policy_version 28960 (0.0009) -[2023-10-15 15:57:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59211776. Throughput: 0: 1779.2, 1: 1794.5. Samples: 14813858. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 15:57:13,441][51532] Avg episode reward: [(0, '33.480'), (1, '36.460')] -[2023-10-15 15:57:14,699][52833] Updated weights for policy 0, policy_version 28870 (0.0009) -[2023-10-15 15:57:15,071][52833] Updated weights for policy 0, policy_version 28880 (0.0007) -[2023-10-15 15:57:15,318][52866] Updated weights for policy 1, policy_version 28970 (0.0008) -[2023-10-15 15:57:15,447][52833] Updated weights for policy 0, policy_version 28890 (0.0009) -[2023-10-15 15:57:15,673][52866] Updated weights for policy 1, policy_version 28980 (0.0007) -[2023-10-15 15:57:16,044][52866] Updated weights for policy 1, policy_version 28990 (0.0007) -[2023-10-15 15:57:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 59277312. Throughput: 0: 1776.0, 1: 1798.1. Samples: 14823644. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-15 15:57:18,442][51532] Avg episode reward: [(0, '33.310'), (1, '36.310')] -[2023-10-15 15:57:19,133][52833] Updated weights for policy 0, policy_version 28900 (0.0008) -[2023-10-15 15:57:19,510][52833] Updated weights for policy 0, policy_version 28910 (0.0008) -[2023-10-15 15:57:19,729][52866] Updated weights for policy 1, policy_version 29000 (0.0008) -[2023-10-15 15:57:19,869][52833] Updated weights for policy 0, policy_version 28920 (0.0009) -[2023-10-15 15:57:20,089][52866] Updated weights for policy 1, policy_version 29010 (0.0008) -[2023-10-15 15:57:20,455][52866] Updated weights for policy 1, policy_version 29020 (0.0008) -[2023-10-15 15:57:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 59342848. Throughput: 0: 1777.9, 1: 1794.0. Samples: 14845870. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 15:57:23,441][51532] Avg episode reward: [(0, '32.060'), (1, '35.930')] -[2023-10-15 15:57:23,806][52833] Updated weights for policy 0, policy_version 28930 (0.0009) -[2023-10-15 15:57:24,214][52833] Updated weights for policy 0, policy_version 28940 (0.0007) -[2023-10-15 15:57:24,279][52866] Updated weights for policy 1, policy_version 29030 (0.0008) -[2023-10-15 15:57:24,573][52833] Updated weights for policy 0, policy_version 28950 (0.0007) -[2023-10-15 15:57:24,645][52866] Updated weights for policy 1, policy_version 29040 (0.0009) -[2023-10-15 15:57:24,944][52833] Updated weights for policy 0, policy_version 28960 (0.0008) -[2023-10-15 15:57:25,007][52866] Updated weights for policy 1, policy_version 29050 (0.0009) -[2023-10-15 15:57:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 59408384. Throughput: 0: 1796.9, 1: 1792.4. Samples: 14868112. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 15:57:28,442][51532] Avg episode reward: [(0, '30.000'), (1, '37.190')] -[2023-10-15 15:57:28,648][52833] Updated weights for policy 0, policy_version 28970 (0.0009) -[2023-10-15 15:57:28,841][52866] Updated weights for policy 1, policy_version 29060 (0.0007) -[2023-10-15 15:57:29,021][52833] Updated weights for policy 0, policy_version 28980 (0.0010) -[2023-10-15 15:57:29,198][52866] Updated weights for policy 1, policy_version 29070 (0.0008) -[2023-10-15 15:57:29,400][52833] Updated weights for policy 0, policy_version 28990 (0.0008) -[2023-10-15 15:57:29,563][52866] Updated weights for policy 1, policy_version 29080 (0.0008) -[2023-10-15 15:57:33,171][52833] Updated weights for policy 0, policy_version 29000 (0.0008) -[2023-10-15 15:57:33,282][52866] Updated weights for policy 1, policy_version 29090 (0.0009) -[2023-10-15 15:57:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 59473920. Throughput: 0: 1775.2, 1: 1791.7. Samples: 14877924. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 15:57:33,442][51532] Avg episode reward: [(0, '30.050'), (1, '37.780')] -[2023-10-15 15:57:33,538][52833] Updated weights for policy 0, policy_version 29010 (0.0008) -[2023-10-15 15:57:33,664][52866] Updated weights for policy 1, policy_version 29100 (0.0008) -[2023-10-15 15:57:33,911][52833] Updated weights for policy 0, policy_version 29020 (0.0009) -[2023-10-15 15:57:34,020][52866] Updated weights for policy 1, policy_version 29110 (0.0008) -[2023-10-15 15:57:34,399][52866] Updated weights for policy 1, policy_version 29120 (0.0009) -[2023-10-15 15:57:37,654][52833] Updated weights for policy 0, policy_version 29030 (0.0007) -[2023-10-15 15:57:38,023][52833] Updated weights for policy 0, policy_version 29040 (0.0007) -[2023-10-15 15:57:38,204][52866] Updated weights for policy 1, policy_version 29130 (0.0008) -[2023-10-15 15:57:38,389][52833] Updated weights for policy 0, policy_version 29050 (0.0007) -[2023-10-15 15:57:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 59539456. Throughput: 0: 1792.8, 1: 1792.3. Samples: 14900298. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 15:57:38,442][51532] Avg episode reward: [(0, '31.720'), (1, '37.480')] -[2023-10-15 15:57:38,570][52866] Updated weights for policy 1, policy_version 29140 (0.0007) -[2023-10-15 15:57:38,939][52866] Updated weights for policy 1, policy_version 29150 (0.0007) -[2023-10-15 15:57:42,032][52833] Updated weights for policy 0, policy_version 29060 (0.0007) -[2023-10-15 15:57:42,405][52833] Updated weights for policy 0, policy_version 29070 (0.0010) -[2023-10-15 15:57:42,690][52866] Updated weights for policy 1, policy_version 29160 (0.0008) -[2023-10-15 15:57:42,764][52833] Updated weights for policy 0, policy_version 29080 (0.0008) -[2023-10-15 15:57:43,053][52866] Updated weights for policy 1, policy_version 29170 (0.0007) -[2023-10-15 15:57:43,410][52866] Updated weights for policy 1, policy_version 29180 (0.0007) -[2023-10-15 15:57:43,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 59637760. Throughput: 0: 1786.5, 1: 1809.2. Samples: 14920996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:57:43,441][51532] Avg episode reward: [(0, '32.400'), (1, '35.750')] -[2023-10-15 15:57:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000029088_29786112.pth... -[2023-10-15 15:57:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000027392_28049408.pth -[2023-10-15 15:57:43,555][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000029184_29884416.pth... -[2023-10-15 15:57:43,597][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000027488_28147712.pth -[2023-10-15 15:57:46,598][52833] Updated weights for policy 0, policy_version 29090 (0.0007) -[2023-10-15 15:57:46,976][52833] Updated weights for policy 0, policy_version 29100 (0.0008) -[2023-10-15 15:57:47,266][52866] Updated weights for policy 1, policy_version 29190 (0.0007) -[2023-10-15 15:57:47,336][52833] Updated weights for policy 0, policy_version 29110 (0.0007) -[2023-10-15 15:57:47,629][52866] Updated weights for policy 1, policy_version 29200 (0.0009) -[2023-10-15 15:57:47,701][52833] Updated weights for policy 0, policy_version 29120 (0.0008) -[2023-10-15 15:57:48,002][52866] Updated weights for policy 1, policy_version 29210 (0.0008) -[2023-10-15 15:57:48,441][51532] Fps is (10 sec: 19660.6, 60 sec: 14746.0, 300 sec: 14440.1). Total num frames: 59736064. Throughput: 0: 1790.7, 1: 1798.0. Samples: 14932634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:57:48,442][51532] Avg episode reward: [(0, '33.140'), (1, '35.550')] -[2023-10-15 15:57:51,339][52833] Updated weights for policy 0, policy_version 29130 (0.0010) -[2023-10-15 15:57:51,702][52833] Updated weights for policy 0, policy_version 29140 (0.0007) -[2023-10-15 15:57:51,759][52866] Updated weights for policy 1, policy_version 29220 (0.0007) -[2023-10-15 15:57:52,072][52833] Updated weights for policy 0, policy_version 29150 (0.0007) -[2023-10-15 15:57:52,115][52866] Updated weights for policy 1, policy_version 29230 (0.0008) -[2023-10-15 15:57:52,489][52866] Updated weights for policy 1, policy_version 29240 (0.0008) -[2023-10-15 15:57:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 59801600. Throughput: 0: 1789.1, 1: 1813.5. Samples: 14953400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:57:53,441][51532] Avg episode reward: [(0, '33.840'), (1, '34.920')] -[2023-10-15 15:57:55,779][52833] Updated weights for policy 0, policy_version 29160 (0.0009) -[2023-10-15 15:57:56,153][52833] Updated weights for policy 0, policy_version 29170 (0.0009) -[2023-10-15 15:57:56,251][52866] Updated weights for policy 1, policy_version 29250 (0.0008) -[2023-10-15 15:57:56,518][52833] Updated weights for policy 0, policy_version 29180 (0.0008) -[2023-10-15 15:57:56,614][52866] Updated weights for policy 1, policy_version 29260 (0.0008) -[2023-10-15 15:57:56,982][52866] Updated weights for policy 1, policy_version 29270 (0.0008) -[2023-10-15 15:57:57,353][52866] Updated weights for policy 1, policy_version 29280 (0.0012) -[2023-10-15 15:57:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59867136. Throughput: 0: 1780.3, 1: 1787.8. Samples: 14974424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:57:58,442][51532] Avg episode reward: [(0, '32.610'), (1, '35.230')] -[2023-10-15 15:58:00,434][52833] Updated weights for policy 0, policy_version 29190 (0.0009) -[2023-10-15 15:58:00,805][52833] Updated weights for policy 0, policy_version 29200 (0.0008) -[2023-10-15 15:58:01,166][52866] Updated weights for policy 1, policy_version 29290 (0.0008) -[2023-10-15 15:58:01,175][52833] Updated weights for policy 0, policy_version 29210 (0.0011) -[2023-10-15 15:58:01,532][52866] Updated weights for policy 1, policy_version 29300 (0.0008) -[2023-10-15 15:58:01,907][52866] Updated weights for policy 1, policy_version 29310 (0.0008) -[2023-10-15 15:58:03,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 59932672. Throughput: 0: 1799.9, 1: 1813.3. Samples: 14986240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:03,443][51532] Avg episode reward: [(0, '34.940'), (1, '36.170')] -[2023-10-15 15:58:04,918][52833] Updated weights for policy 0, policy_version 29220 (0.0007) -[2023-10-15 15:58:05,278][52833] Updated weights for policy 0, policy_version 29230 (0.0008) -[2023-10-15 15:58:05,507][52866] Updated weights for policy 1, policy_version 29320 (0.0008) -[2023-10-15 15:58:05,644][52833] Updated weights for policy 0, policy_version 29240 (0.0007) -[2023-10-15 15:58:05,875][52866] Updated weights for policy 1, policy_version 29330 (0.0008) -[2023-10-15 15:58:06,246][52866] Updated weights for policy 1, policy_version 29340 (0.0008) -[2023-10-15 15:58:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 59998208. Throughput: 0: 1784.3, 1: 1794.9. Samples: 15006934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:08,442][51532] Avg episode reward: [(0, '36.030'), (1, '40.230')] -[2023-10-15 15:58:08,443][52518] Saving new best policy, reward=40.230! -[2023-10-15 15:58:09,362][52833] Updated weights for policy 0, policy_version 29250 (0.0007) -[2023-10-15 15:58:09,763][52833] Updated weights for policy 0, policy_version 29260 (0.0008) -[2023-10-15 15:58:10,018][52866] Updated weights for policy 1, policy_version 29350 (0.0007) -[2023-10-15 15:58:10,125][52833] Updated weights for policy 0, policy_version 29270 (0.0008) -[2023-10-15 15:58:10,380][52866] Updated weights for policy 1, policy_version 29360 (0.0007) -[2023-10-15 15:58:10,486][52833] Updated weights for policy 0, policy_version 29280 (0.0008) -[2023-10-15 15:58:10,742][52866] Updated weights for policy 1, policy_version 29370 (0.0007) -[2023-10-15 15:58:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 60063744. Throughput: 0: 1785.4, 1: 1800.0. Samples: 15029458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:13,442][51532] Avg episode reward: [(0, '33.970'), (1, '39.170')] -[2023-10-15 15:58:14,287][52833] Updated weights for policy 0, policy_version 29290 (0.0008) -[2023-10-15 15:58:14,373][52866] Updated weights for policy 1, policy_version 29380 (0.0007) -[2023-10-15 15:58:14,653][52833] Updated weights for policy 0, policy_version 29300 (0.0008) -[2023-10-15 15:58:14,745][52866] Updated weights for policy 1, policy_version 29390 (0.0007) -[2023-10-15 15:58:15,020][52833] Updated weights for policy 0, policy_version 29310 (0.0008) -[2023-10-15 15:58:15,107][52866] Updated weights for policy 1, policy_version 29400 (0.0008) -[2023-10-15 15:58:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 60129280. Throughput: 0: 1786.2, 1: 1798.5. Samples: 15039236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:18,442][51532] Avg episode reward: [(0, '34.910'), (1, '39.520')] -[2023-10-15 15:58:18,748][52833] Updated weights for policy 0, policy_version 29320 (0.0008) -[2023-10-15 15:58:18,862][52866] Updated weights for policy 1, policy_version 29410 (0.0009) -[2023-10-15 15:58:19,115][52833] Updated weights for policy 0, policy_version 29330 (0.0008) -[2023-10-15 15:58:19,226][52866] Updated weights for policy 1, policy_version 29420 (0.0009) -[2023-10-15 15:58:19,483][52833] Updated weights for policy 0, policy_version 29340 (0.0008) -[2023-10-15 15:58:19,594][52866] Updated weights for policy 1, policy_version 29430 (0.0008) -[2023-10-15 15:58:19,957][52866] Updated weights for policy 1, policy_version 29440 (0.0009) -[2023-10-15 15:58:23,146][52833] Updated weights for policy 0, policy_version 29350 (0.0009) -[2023-10-15 15:58:23,441][51532] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 60194816. Throughput: 0: 1784.8, 1: 1800.7. Samples: 15061646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:23,441][51532] Avg episode reward: [(0, '33.490'), (1, '40.570')] -[2023-10-15 15:58:23,509][52833] Updated weights for policy 0, policy_version 29360 (0.0009) -[2023-10-15 15:58:23,719][52866] Updated weights for policy 1, policy_version 29450 (0.0008) -[2023-10-15 15:58:23,874][52833] Updated weights for policy 0, policy_version 29370 (0.0008) -[2023-10-15 15:58:24,075][52866] Updated weights for policy 1, policy_version 29460 (0.0008) -[2023-10-15 15:58:24,448][52866] Updated weights for policy 1, policy_version 29470 (0.0009) -[2023-10-15 15:58:24,517][52518] Saving new best policy, reward=40.570! -[2023-10-15 15:58:27,752][52833] Updated weights for policy 0, policy_version 29380 (0.0007) -[2023-10-15 15:58:28,116][52833] Updated weights for policy 0, policy_version 29390 (0.0010) -[2023-10-15 15:58:28,144][52866] Updated weights for policy 1, policy_version 29480 (0.0008) -[2023-10-15 15:58:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 60260352. Throughput: 0: 1799.4, 1: 1815.6. Samples: 15083672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:28,441][51532] Avg episode reward: [(0, '33.350'), (1, '41.480')] -[2023-10-15 15:58:28,483][52833] Updated weights for policy 0, policy_version 29400 (0.0011) -[2023-10-15 15:58:28,519][52866] Updated weights for policy 1, policy_version 29490 (0.0008) -[2023-10-15 15:58:28,891][52866] Updated weights for policy 1, policy_version 29500 (0.0008) -[2023-10-15 15:58:29,027][52518] Saving new best policy, reward=41.480! -[2023-10-15 15:58:32,241][52833] Updated weights for policy 0, policy_version 29410 (0.0008) -[2023-10-15 15:58:32,600][52866] Updated weights for policy 1, policy_version 29510 (0.0007) -[2023-10-15 15:58:32,620][52833] Updated weights for policy 0, policy_version 29420 (0.0007) -[2023-10-15 15:58:32,955][52866] Updated weights for policy 1, policy_version 29520 (0.0007) -[2023-10-15 15:58:32,993][52833] Updated weights for policy 0, policy_version 29430 (0.0007) -[2023-10-15 15:58:33,329][52866] Updated weights for policy 1, policy_version 29530 (0.0007) -[2023-10-15 15:58:33,353][52833] Updated weights for policy 0, policy_version 29440 (0.0008) -[2023-10-15 15:58:33,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 60358656. Throughput: 0: 1781.2, 1: 1801.0. Samples: 15093832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:33,442][51532] Avg episode reward: [(0, '33.380'), (1, '39.500')] -[2023-10-15 15:58:37,022][52866] Updated weights for policy 1, policy_version 29540 (0.0009) -[2023-10-15 15:58:37,241][52833] Updated weights for policy 0, policy_version 29450 (0.0008) -[2023-10-15 15:58:37,398][52866] Updated weights for policy 1, policy_version 29550 (0.0009) -[2023-10-15 15:58:37,599][52833] Updated weights for policy 0, policy_version 29460 (0.0011) -[2023-10-15 15:58:37,755][52866] Updated weights for policy 1, policy_version 29560 (0.0008) -[2023-10-15 15:58:37,967][52833] Updated weights for policy 0, policy_version 29470 (0.0010) -[2023-10-15 15:58:38,441][51532] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 60456960. Throughput: 0: 1805.1, 1: 1806.4. Samples: 15115920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:38,442][51532] Avg episode reward: [(0, '33.180'), (1, '39.470')] -[2023-10-15 15:58:41,665][52833] Updated weights for policy 0, policy_version 29480 (0.0009) -[2023-10-15 15:58:41,679][52866] Updated weights for policy 1, policy_version 29570 (0.0007) -[2023-10-15 15:58:42,036][52833] Updated weights for policy 0, policy_version 29490 (0.0007) -[2023-10-15 15:58:42,040][52866] Updated weights for policy 1, policy_version 29580 (0.0007) -[2023-10-15 15:58:42,399][52833] Updated weights for policy 0, policy_version 29500 (0.0007) -[2023-10-15 15:58:42,404][52866] Updated weights for policy 1, policy_version 29590 (0.0007) -[2023-10-15 15:58:42,780][52866] Updated weights for policy 1, policy_version 29600 (0.0010) -[2023-10-15 15:58:43,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 60522496. Throughput: 0: 1782.4, 1: 1794.6. Samples: 15135390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:43,442][51532] Avg episode reward: [(0, '32.120'), (1, '37.310')] -[2023-10-15 15:58:46,034][52833] Updated weights for policy 0, policy_version 29510 (0.0008) -[2023-10-15 15:58:46,403][52833] Updated weights for policy 0, policy_version 29520 (0.0007) -[2023-10-15 15:58:46,693][52866] Updated weights for policy 1, policy_version 29610 (0.0008) -[2023-10-15 15:58:46,774][52833] Updated weights for policy 0, policy_version 29530 (0.0010) -[2023-10-15 15:58:47,055][52866] Updated weights for policy 1, policy_version 29620 (0.0010) -[2023-10-15 15:58:47,420][52866] Updated weights for policy 1, policy_version 29630 (0.0010) -[2023-10-15 15:58:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60588032. Throughput: 0: 1805.2, 1: 1799.1. Samples: 15148430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:58:48,441][51532] Avg episode reward: [(0, '34.790'), (1, '33.300')] -[2023-10-15 15:58:50,563][52833] Updated weights for policy 0, policy_version 29540 (0.0008) -[2023-10-15 15:58:50,928][52833] Updated weights for policy 0, policy_version 29550 (0.0009) -[2023-10-15 15:58:51,215][52866] Updated weights for policy 1, policy_version 29640 (0.0008) -[2023-10-15 15:58:51,296][52833] Updated weights for policy 0, policy_version 29560 (0.0009) -[2023-10-15 15:58:51,585][52866] Updated weights for policy 1, policy_version 29650 (0.0007) -[2023-10-15 15:58:51,956][52866] Updated weights for policy 1, policy_version 29660 (0.0009) -[2023-10-15 15:58:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 60653568. Throughput: 0: 1789.1, 1: 1786.8. Samples: 15167850. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:58:53,442][51532] Avg episode reward: [(0, '33.990'), (1, '32.450')] -[2023-10-15 15:58:54,912][52833] Updated weights for policy 0, policy_version 29570 (0.0007) -[2023-10-15 15:58:55,305][52833] Updated weights for policy 0, policy_version 29580 (0.0009) -[2023-10-15 15:58:55,676][52833] Updated weights for policy 0, policy_version 29590 (0.0008) -[2023-10-15 15:58:55,680][52866] Updated weights for policy 1, policy_version 29670 (0.0009) -[2023-10-15 15:58:56,039][52833] Updated weights for policy 0, policy_version 29600 (0.0008) -[2023-10-15 15:58:56,043][52866] Updated weights for policy 1, policy_version 29680 (0.0009) -[2023-10-15 15:58:56,415][52866] Updated weights for policy 1, policy_version 29690 (0.0007) -[2023-10-15 15:58:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60719104. Throughput: 0: 1797.5, 1: 1776.0. Samples: 15190262. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:58:58,442][51532] Avg episode reward: [(0, '35.030'), (1, '31.960')] -[2023-10-15 15:58:59,724][52833] Updated weights for policy 0, policy_version 29610 (0.0010) -[2023-10-15 15:59:00,092][52833] Updated weights for policy 0, policy_version 29620 (0.0008) -[2023-10-15 15:59:00,188][52866] Updated weights for policy 1, policy_version 29700 (0.0008) -[2023-10-15 15:59:00,471][52833] Updated weights for policy 0, policy_version 29630 (0.0009) -[2023-10-15 15:59:00,550][52866] Updated weights for policy 1, policy_version 29710 (0.0007) -[2023-10-15 15:59:00,925][52866] Updated weights for policy 1, policy_version 29720 (0.0009) -[2023-10-15 15:59:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 60784640. Throughput: 0: 1798.2, 1: 1783.0. Samples: 15200388. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:59:03,442][51532] Avg episode reward: [(0, '36.230'), (1, '32.000')] -[2023-10-15 15:59:04,335][52833] Updated weights for policy 0, policy_version 29640 (0.0008) -[2023-10-15 15:59:04,588][52866] Updated weights for policy 1, policy_version 29730 (0.0008) -[2023-10-15 15:59:04,708][52833] Updated weights for policy 0, policy_version 29650 (0.0009) -[2023-10-15 15:59:04,949][52866] Updated weights for policy 1, policy_version 29740 (0.0007) -[2023-10-15 15:59:05,077][52833] Updated weights for policy 0, policy_version 29660 (0.0009) -[2023-10-15 15:59:05,319][52866] Updated weights for policy 1, policy_version 29750 (0.0007) -[2023-10-15 15:59:05,688][52866] Updated weights for policy 1, policy_version 29760 (0.0008) -[2023-10-15 15:59:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 60850176. Throughput: 0: 1797.8, 1: 1779.4. Samples: 15222620. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:59:08,442][51532] Avg episode reward: [(0, '38.820'), (1, '32.060')] -[2023-10-15 15:59:08,785][52833] Updated weights for policy 0, policy_version 29670 (0.0009) -[2023-10-15 15:59:09,154][52833] Updated weights for policy 0, policy_version 29680 (0.0007) -[2023-10-15 15:59:09,528][52833] Updated weights for policy 0, policy_version 29690 (0.0010) -[2023-10-15 15:59:09,560][52866] Updated weights for policy 1, policy_version 29770 (0.0007) -[2023-10-15 15:59:09,922][52866] Updated weights for policy 1, policy_version 29780 (0.0008) -[2023-10-15 15:59:10,287][52866] Updated weights for policy 1, policy_version 29790 (0.0008) -[2023-10-15 15:59:13,278][52833] Updated weights for policy 0, policy_version 29700 (0.0008) -[2023-10-15 15:59:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 60915712. Throughput: 0: 1805.0, 1: 1776.7. Samples: 15244850. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) -[2023-10-15 15:59:13,442][51532] Avg episode reward: [(0, '37.660'), (1, '33.300')] -[2023-10-15 15:59:13,653][52833] Updated weights for policy 0, policy_version 29710 (0.0010) -[2023-10-15 15:59:14,015][52833] Updated weights for policy 0, policy_version 29720 (0.0008) -[2023-10-15 15:59:14,047][52866] Updated weights for policy 1, policy_version 29800 (0.0007) -[2023-10-15 15:59:14,416][52866] Updated weights for policy 1, policy_version 29810 (0.0007) -[2023-10-15 15:59:14,780][52866] Updated weights for policy 1, policy_version 29820 (0.0009) -[2023-10-15 15:59:17,791][52833] Updated weights for policy 0, policy_version 29730 (0.0008) -[2023-10-15 15:59:18,156][52833] Updated weights for policy 0, policy_version 29740 (0.0008) -[2023-10-15 15:59:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 60981248. Throughput: 0: 1799.6, 1: 1775.6. Samples: 15254718. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) -[2023-10-15 15:59:18,442][51532] Avg episode reward: [(0, '37.950'), (1, '37.660')] -[2023-10-15 15:59:18,520][52833] Updated weights for policy 0, policy_version 29750 (0.0007) -[2023-10-15 15:59:18,574][52866] Updated weights for policy 1, policy_version 29830 (0.0008) -[2023-10-15 15:59:18,899][52833] Updated weights for policy 0, policy_version 29760 (0.0008) -[2023-10-15 15:59:18,928][52866] Updated weights for policy 1, policy_version 29840 (0.0009) -[2023-10-15 15:59:19,294][52866] Updated weights for policy 1, policy_version 29850 (0.0007) -[2023-10-15 15:59:22,614][52833] Updated weights for policy 0, policy_version 29770 (0.0008) -[2023-10-15 15:59:22,991][52833] Updated weights for policy 0, policy_version 29780 (0.0008) -[2023-10-15 15:59:23,183][52866] Updated weights for policy 1, policy_version 29860 (0.0008) -[2023-10-15 15:59:23,357][52833] Updated weights for policy 0, policy_version 29790 (0.0009) -[2023-10-15 15:59:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 61079552. Throughput: 0: 1800.2, 1: 1778.9. Samples: 15276982. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) -[2023-10-15 15:59:23,442][51532] Avg episode reward: [(0, '40.370'), (1, '35.690')] -[2023-10-15 15:59:23,442][52410] Saving new best policy, reward=40.370! -[2023-10-15 15:59:23,542][52866] Updated weights for policy 1, policy_version 29870 (0.0007) -[2023-10-15 15:59:23,910][52866] Updated weights for policy 1, policy_version 29880 (0.0010) -[2023-10-15 15:59:27,136][52833] Updated weights for policy 0, policy_version 29800 (0.0007) -[2023-10-15 15:59:27,506][52833] Updated weights for policy 0, policy_version 29810 (0.0009) -[2023-10-15 15:59:27,541][52866] Updated weights for policy 1, policy_version 29890 (0.0008) -[2023-10-15 15:59:27,865][52833] Updated weights for policy 0, policy_version 29820 (0.0009) -[2023-10-15 15:59:27,914][52866] Updated weights for policy 1, policy_version 29900 (0.0007) -[2023-10-15 15:59:28,276][52866] Updated weights for policy 1, policy_version 29910 (0.0007) -[2023-10-15 15:59:28,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 61145088. Throughput: 0: 1806.1, 1: 1806.6. Samples: 15297962. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) -[2023-10-15 15:59:28,441][51532] Avg episode reward: [(0, '43.410'), (1, '35.610')] -[2023-10-15 15:59:28,448][52410] Saving new best policy, reward=43.410! -[2023-10-15 15:59:28,647][52866] Updated weights for policy 1, policy_version 29920 (0.0008) -[2023-10-15 15:59:31,618][52833] Updated weights for policy 0, policy_version 29830 (0.0007) -[2023-10-15 15:59:31,990][52833] Updated weights for policy 0, policy_version 29840 (0.0010) -[2023-10-15 15:59:32,357][52833] Updated weights for policy 0, policy_version 29850 (0.0008) -[2023-10-15 15:59:32,556][52866] Updated weights for policy 1, policy_version 29930 (0.0007) -[2023-10-15 15:59:32,920][52866] Updated weights for policy 1, policy_version 29940 (0.0007) -[2023-10-15 15:59:33,280][52866] Updated weights for policy 1, policy_version 29950 (0.0009) -[2023-10-15 15:59:33,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 61243392. Throughput: 0: 1795.5, 1: 1788.0. Samples: 15309686. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) -[2023-10-15 15:59:33,441][51532] Avg episode reward: [(0, '41.470'), (1, '35.250')] -[2023-10-15 15:59:35,916][52833] Updated weights for policy 0, policy_version 29860 (0.0008) -[2023-10-15 15:59:36,276][52833] Updated weights for policy 0, policy_version 29870 (0.0008) -[2023-10-15 15:59:36,654][52833] Updated weights for policy 0, policy_version 29880 (0.0009) -[2023-10-15 15:59:37,050][52866] Updated weights for policy 1, policy_version 29960 (0.0008) -[2023-10-15 15:59:37,415][52866] Updated weights for policy 1, policy_version 29970 (0.0007) -[2023-10-15 15:59:37,782][52866] Updated weights for policy 1, policy_version 29980 (0.0007) -[2023-10-15 15:59:38,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 61308928. Throughput: 0: 1801.9, 1: 1810.8. Samples: 15330422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:59:38,442][51532] Avg episode reward: [(0, '42.060'), (1, '37.800')] -[2023-10-15 15:59:40,308][52833] Updated weights for policy 0, policy_version 29890 (0.0007) -[2023-10-15 15:59:40,695][52833] Updated weights for policy 0, policy_version 29900 (0.0007) -[2023-10-15 15:59:41,064][52833] Updated weights for policy 0, policy_version 29910 (0.0008) -[2023-10-15 15:59:41,427][52833] Updated weights for policy 0, policy_version 29920 (0.0009) -[2023-10-15 15:59:41,433][52866] Updated weights for policy 1, policy_version 29990 (0.0008) -[2023-10-15 15:59:41,803][52866] Updated weights for policy 1, policy_version 30000 (0.0007) -[2023-10-15 15:59:42,164][52866] Updated weights for policy 1, policy_version 30010 (0.0007) -[2023-10-15 15:59:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61374464. Throughput: 0: 1795.1, 1: 1796.2. Samples: 15351872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:59:43,442][51532] Avg episode reward: [(0, '36.940'), (1, '38.720')] -[2023-10-15 15:59:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000029920_30638080.pth... -[2023-10-15 15:59:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000030016_30736384.pth... -[2023-10-15 15:59:43,482][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000028256_28934144.pth -[2023-10-15 15:59:43,492][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000028320_28999680.pth -[2023-10-15 15:59:45,191][52833] Updated weights for policy 0, policy_version 29930 (0.0011) -[2023-10-15 15:59:45,556][52833] Updated weights for policy 0, policy_version 29940 (0.0009) -[2023-10-15 15:59:45,763][52866] Updated weights for policy 1, policy_version 30020 (0.0009) -[2023-10-15 15:59:45,918][52833] Updated weights for policy 0, policy_version 29950 (0.0007) -[2023-10-15 15:59:46,134][52866] Updated weights for policy 1, policy_version 30030 (0.0009) -[2023-10-15 15:59:46,501][52866] Updated weights for policy 1, policy_version 30040 (0.0009) -[2023-10-15 15:59:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61440000. Throughput: 0: 1797.6, 1: 1813.0. Samples: 15362866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:59:48,442][51532] Avg episode reward: [(0, '35.490'), (1, '37.210')] -[2023-10-15 15:59:49,774][52833] Updated weights for policy 0, policy_version 29960 (0.0009) -[2023-10-15 15:59:50,146][52833] Updated weights for policy 0, policy_version 29970 (0.0010) -[2023-10-15 15:59:50,223][52866] Updated weights for policy 1, policy_version 30050 (0.0009) -[2023-10-15 15:59:50,523][52833] Updated weights for policy 0, policy_version 29980 (0.0008) -[2023-10-15 15:59:50,587][52866] Updated weights for policy 1, policy_version 30060 (0.0009) -[2023-10-15 15:59:50,952][52866] Updated weights for policy 1, policy_version 30070 (0.0010) -[2023-10-15 15:59:51,318][52866] Updated weights for policy 1, policy_version 30080 (0.0010) -[2023-10-15 15:59:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 61505536. Throughput: 0: 1792.3, 1: 1793.2. Samples: 15383968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:59:53,442][51532] Avg episode reward: [(0, '38.130'), (1, '38.090')] -[2023-10-15 15:59:54,260][52833] Updated weights for policy 0, policy_version 29990 (0.0011) -[2023-10-15 15:59:54,632][52833] Updated weights for policy 0, policy_version 30000 (0.0008) -[2023-10-15 15:59:54,955][52866] Updated weights for policy 1, policy_version 30090 (0.0009) -[2023-10-15 15:59:54,997][52833] Updated weights for policy 0, policy_version 30010 (0.0007) -[2023-10-15 15:59:55,323][52866] Updated weights for policy 1, policy_version 30100 (0.0010) -[2023-10-15 15:59:55,686][52866] Updated weights for policy 1, policy_version 30110 (0.0009) -[2023-10-15 15:59:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 61571072. Throughput: 0: 1797.4, 1: 1796.5. Samples: 15406578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 15:59:58,442][51532] Avg episode reward: [(0, '35.800'), (1, '34.020')] -[2023-10-15 15:59:58,619][52833] Updated weights for policy 0, policy_version 30020 (0.0007) -[2023-10-15 15:59:58,988][52833] Updated weights for policy 0, policy_version 30030 (0.0007) -[2023-10-15 15:59:59,362][52833] Updated weights for policy 0, policy_version 30040 (0.0008) -[2023-10-15 15:59:59,434][52866] Updated weights for policy 1, policy_version 30120 (0.0008) -[2023-10-15 15:59:59,809][52866] Updated weights for policy 1, policy_version 30130 (0.0009) -[2023-10-15 16:00:00,178][52866] Updated weights for policy 1, policy_version 30140 (0.0010) -[2023-10-15 16:00:03,289][52833] Updated weights for policy 0, policy_version 30050 (0.0009) -[2023-10-15 16:00:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 61636608. Throughput: 0: 1796.4, 1: 1792.9. Samples: 15416240. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:00:03,442][51532] Avg episode reward: [(0, '33.540'), (1, '33.750')] -[2023-10-15 16:00:03,665][52833] Updated weights for policy 0, policy_version 30060 (0.0007) -[2023-10-15 16:00:03,948][52866] Updated weights for policy 1, policy_version 30150 (0.0008) -[2023-10-15 16:00:04,040][52833] Updated weights for policy 0, policy_version 30070 (0.0009) -[2023-10-15 16:00:04,308][52866] Updated weights for policy 1, policy_version 30160 (0.0008) -[2023-10-15 16:00:04,399][52833] Updated weights for policy 0, policy_version 30080 (0.0007) -[2023-10-15 16:00:04,686][52866] Updated weights for policy 1, policy_version 30170 (0.0009) -[2023-10-15 16:00:08,180][52833] Updated weights for policy 0, policy_version 30090 (0.0008) -[2023-10-15 16:00:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 61702144. Throughput: 0: 1798.7, 1: 1795.1. Samples: 15438706. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:00:08,442][51532] Avg episode reward: [(0, '33.480'), (1, '37.040')] -[2023-10-15 16:00:08,534][52866] Updated weights for policy 1, policy_version 30180 (0.0007) -[2023-10-15 16:00:08,547][52833] Updated weights for policy 0, policy_version 30100 (0.0008) -[2023-10-15 16:00:08,903][52866] Updated weights for policy 1, policy_version 30190 (0.0007) -[2023-10-15 16:00:08,918][52833] Updated weights for policy 0, policy_version 30110 (0.0007) -[2023-10-15 16:00:09,272][52866] Updated weights for policy 1, policy_version 30200 (0.0008) -[2023-10-15 16:00:12,904][52833] Updated weights for policy 0, policy_version 30120 (0.0009) -[2023-10-15 16:00:13,067][52866] Updated weights for policy 1, policy_version 30210 (0.0008) -[2023-10-15 16:00:13,272][52833] Updated weights for policy 0, policy_version 30130 (0.0010) -[2023-10-15 16:00:13,435][52866] Updated weights for policy 1, policy_version 30220 (0.0008) -[2023-10-15 16:00:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 61767680. Throughput: 0: 1810.6, 1: 1809.0. Samples: 15460846. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:00:13,441][51532] Avg episode reward: [(0, '34.690'), (1, '37.540')] -[2023-10-15 16:00:13,644][52833] Updated weights for policy 0, policy_version 30140 (0.0009) -[2023-10-15 16:00:13,790][52866] Updated weights for policy 1, policy_version 30230 (0.0007) -[2023-10-15 16:00:14,159][52866] Updated weights for policy 1, policy_version 30240 (0.0008) -[2023-10-15 16:00:17,456][52833] Updated weights for policy 0, policy_version 30150 (0.0008) -[2023-10-15 16:00:17,822][52833] Updated weights for policy 0, policy_version 30160 (0.0008) -[2023-10-15 16:00:17,920][52866] Updated weights for policy 1, policy_version 30250 (0.0008) -[2023-10-15 16:00:18,194][52833] Updated weights for policy 0, policy_version 30170 (0.0007) -[2023-10-15 16:00:18,287][52866] Updated weights for policy 1, policy_version 30260 (0.0008) -[2023-10-15 16:00:18,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 61865984. Throughput: 0: 1786.7, 1: 1796.4. Samples: 15470922. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:00:18,441][51532] Avg episode reward: [(0, '34.950'), (1, '38.570')] -[2023-10-15 16:00:18,652][52866] Updated weights for policy 1, policy_version 30270 (0.0007) -[2023-10-15 16:00:21,938][52833] Updated weights for policy 0, policy_version 30180 (0.0008) -[2023-10-15 16:00:22,293][52833] Updated weights for policy 0, policy_version 30190 (0.0009) -[2023-10-15 16:00:22,413][52866] Updated weights for policy 1, policy_version 30280 (0.0007) -[2023-10-15 16:00:22,671][52833] Updated weights for policy 0, policy_version 30200 (0.0008) -[2023-10-15 16:00:22,780][52866] Updated weights for policy 1, policy_version 30290 (0.0008) -[2023-10-15 16:00:23,150][52866] Updated weights for policy 1, policy_version 30300 (0.0009) -[2023-10-15 16:00:23,441][51532] Fps is (10 sec: 19660.1, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 61964288. Throughput: 0: 1808.0, 1: 1804.1. Samples: 15492968. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) -[2023-10-15 16:00:23,443][51532] Avg episode reward: [(0, '35.650'), (1, '36.450')] -[2023-10-15 16:00:26,593][52833] Updated weights for policy 0, policy_version 30210 (0.0007) -[2023-10-15 16:00:27,005][52833] Updated weights for policy 0, policy_version 30220 (0.0008) -[2023-10-15 16:00:27,017][52866] Updated weights for policy 1, policy_version 30310 (0.0009) -[2023-10-15 16:00:27,369][52833] Updated weights for policy 0, policy_version 30230 (0.0008) -[2023-10-15 16:00:27,384][52866] Updated weights for policy 1, policy_version 30320 (0.0008) -[2023-10-15 16:00:27,737][52833] Updated weights for policy 0, policy_version 30240 (0.0008) -[2023-10-15 16:00:27,748][52866] Updated weights for policy 1, policy_version 30330 (0.0009) -[2023-10-15 16:00:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 62029824. Throughput: 0: 1772.8, 1: 1791.7. Samples: 15512272. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) -[2023-10-15 16:00:28,441][51532] Avg episode reward: [(0, '36.640'), (1, '35.470')] -[2023-10-15 16:00:31,465][52833] Updated weights for policy 0, policy_version 30250 (0.0009) -[2023-10-15 16:00:31,512][52866] Updated weights for policy 1, policy_version 30340 (0.0009) -[2023-10-15 16:00:31,838][52833] Updated weights for policy 0, policy_version 30260 (0.0008) -[2023-10-15 16:00:31,875][52866] Updated weights for policy 1, policy_version 30350 (0.0007) -[2023-10-15 16:00:32,203][52833] Updated weights for policy 0, policy_version 30270 (0.0007) -[2023-10-15 16:00:32,235][52866] Updated weights for policy 1, policy_version 30360 (0.0007) -[2023-10-15 16:00:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 62095360. Throughput: 0: 1801.4, 1: 1799.1. Samples: 15524886. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) -[2023-10-15 16:00:33,443][51532] Avg episode reward: [(0, '36.560'), (1, '32.770')] -[2023-10-15 16:00:35,884][52833] Updated weights for policy 0, policy_version 30280 (0.0008) -[2023-10-15 16:00:36,020][52866] Updated weights for policy 1, policy_version 30370 (0.0007) -[2023-10-15 16:00:36,251][52833] Updated weights for policy 0, policy_version 30290 (0.0008) -[2023-10-15 16:00:36,376][52866] Updated weights for policy 1, policy_version 30380 (0.0008) -[2023-10-15 16:00:36,610][52833] Updated weights for policy 0, policy_version 30300 (0.0008) -[2023-10-15 16:00:36,747][52866] Updated weights for policy 1, policy_version 30390 (0.0009) -[2023-10-15 16:00:37,108][52866] Updated weights for policy 1, policy_version 30400 (0.0008) -[2023-10-15 16:00:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62160896. Throughput: 0: 1774.8, 1: 1792.9. Samples: 15544514. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) -[2023-10-15 16:00:38,441][51532] Avg episode reward: [(0, '38.640'), (1, '34.290')] -[2023-10-15 16:00:40,374][52833] Updated weights for policy 0, policy_version 30310 (0.0008) -[2023-10-15 16:00:40,735][52833] Updated weights for policy 0, policy_version 30320 (0.0008) -[2023-10-15 16:00:40,760][52866] Updated weights for policy 1, policy_version 30410 (0.0007) -[2023-10-15 16:00:41,108][52833] Updated weights for policy 0, policy_version 30330 (0.0010) -[2023-10-15 16:00:41,128][52866] Updated weights for policy 1, policy_version 30420 (0.0007) -[2023-10-15 16:00:41,497][52866] Updated weights for policy 1, policy_version 30430 (0.0010) -[2023-10-15 16:00:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62226432. Throughput: 0: 1776.8, 1: 1787.7. Samples: 15566980. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) -[2023-10-15 16:00:43,442][51532] Avg episode reward: [(0, '36.860'), (1, '37.660')] -[2023-10-15 16:00:44,776][52833] Updated weights for policy 0, policy_version 30340 (0.0008) -[2023-10-15 16:00:45,145][52833] Updated weights for policy 0, policy_version 30350 (0.0007) -[2023-10-15 16:00:45,303][52866] Updated weights for policy 1, policy_version 30440 (0.0008) -[2023-10-15 16:00:45,517][52833] Updated weights for policy 0, policy_version 30360 (0.0007) -[2023-10-15 16:00:45,670][52866] Updated weights for policy 1, policy_version 30450 (0.0007) -[2023-10-15 16:00:46,027][52866] Updated weights for policy 1, policy_version 30460 (0.0008) -[2023-10-15 16:00:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 62291968. Throughput: 0: 1775.6, 1: 1797.8. Samples: 15577046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:00:48,442][51532] Avg episode reward: [(0, '36.300'), (1, '38.250')] -[2023-10-15 16:00:49,395][52833] Updated weights for policy 0, policy_version 30370 (0.0009) -[2023-10-15 16:00:49,757][52833] Updated weights for policy 0, policy_version 30380 (0.0009) -[2023-10-15 16:00:49,865][52866] Updated weights for policy 1, policy_version 30470 (0.0008) -[2023-10-15 16:00:50,125][52833] Updated weights for policy 0, policy_version 30390 (0.0007) -[2023-10-15 16:00:50,222][52866] Updated weights for policy 1, policy_version 30480 (0.0007) -[2023-10-15 16:00:50,498][52833] Updated weights for policy 0, policy_version 30400 (0.0009) -[2023-10-15 16:00:50,596][52866] Updated weights for policy 1, policy_version 30490 (0.0007) -[2023-10-15 16:00:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14329.1). Total num frames: 62357504. Throughput: 0: 1777.4, 1: 1787.6. Samples: 15599130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:00:53,441][51532] Avg episode reward: [(0, '35.080'), (1, '37.410')] -[2023-10-15 16:00:54,136][52833] Updated weights for policy 0, policy_version 30410 (0.0012) -[2023-10-15 16:00:54,326][52866] Updated weights for policy 1, policy_version 30500 (0.0008) -[2023-10-15 16:00:54,505][52833] Updated weights for policy 0, policy_version 30420 (0.0009) -[2023-10-15 16:00:54,703][52866] Updated weights for policy 1, policy_version 30510 (0.0007) -[2023-10-15 16:00:54,883][52833] Updated weights for policy 0, policy_version 30430 (0.0008) -[2023-10-15 16:00:55,065][52866] Updated weights for policy 1, policy_version 30520 (0.0009) -[2023-10-15 16:00:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 62423040. Throughput: 0: 1792.8, 1: 1778.2. Samples: 15621538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:00:58,442][51532] Avg episode reward: [(0, '34.910'), (1, '38.650')] -[2023-10-15 16:00:58,542][52833] Updated weights for policy 0, policy_version 30440 (0.0009) -[2023-10-15 16:00:58,890][52866] Updated weights for policy 1, policy_version 30530 (0.0008) -[2023-10-15 16:00:58,918][52833] Updated weights for policy 0, policy_version 30450 (0.0008) -[2023-10-15 16:00:59,259][52866] Updated weights for policy 1, policy_version 30540 (0.0009) -[2023-10-15 16:00:59,279][52833] Updated weights for policy 0, policy_version 30460 (0.0008) -[2023-10-15 16:00:59,624][52866] Updated weights for policy 1, policy_version 30550 (0.0008) -[2023-10-15 16:00:59,988][52866] Updated weights for policy 1, policy_version 30560 (0.0008) -[2023-10-15 16:01:03,102][52833] Updated weights for policy 0, policy_version 30470 (0.0009) -[2023-10-15 16:01:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 62488576. Throughput: 0: 1789.9, 1: 1776.5. Samples: 15631410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:01:03,441][51532] Avg episode reward: [(0, '34.020'), (1, '37.670')] -[2023-10-15 16:01:03,466][52833] Updated weights for policy 0, policy_version 30480 (0.0010) -[2023-10-15 16:01:03,834][52833] Updated weights for policy 0, policy_version 30490 (0.0007) -[2023-10-15 16:01:03,851][52866] Updated weights for policy 1, policy_version 30570 (0.0008) -[2023-10-15 16:01:04,220][52866] Updated weights for policy 1, policy_version 30580 (0.0008) -[2023-10-15 16:01:04,577][52866] Updated weights for policy 1, policy_version 30590 (0.0009) -[2023-10-15 16:01:07,387][52833] Updated weights for policy 0, policy_version 30500 (0.0009) -[2023-10-15 16:01:07,751][52833] Updated weights for policy 0, policy_version 30510 (0.0007) -[2023-10-15 16:01:08,120][52833] Updated weights for policy 0, policy_version 30520 (0.0008) -[2023-10-15 16:01:08,362][52866] Updated weights for policy 1, policy_version 30600 (0.0008) -[2023-10-15 16:01:08,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 62586880. Throughput: 0: 1792.5, 1: 1773.2. Samples: 15653424. Policy #0 lag: (min: 34.0, avg: 47.3, max: 48.0) -[2023-10-15 16:01:08,442][51532] Avg episode reward: [(0, '32.640'), (1, '38.730')] -[2023-10-15 16:01:08,726][52866] Updated weights for policy 1, policy_version 30610 (0.0008) -[2023-10-15 16:01:09,095][52866] Updated weights for policy 1, policy_version 30620 (0.0008) -[2023-10-15 16:01:11,965][52833] Updated weights for policy 0, policy_version 30530 (0.0008) -[2023-10-15 16:01:12,346][52833] Updated weights for policy 0, policy_version 30540 (0.0008) -[2023-10-15 16:01:12,708][52833] Updated weights for policy 0, policy_version 30550 (0.0009) -[2023-10-15 16:01:12,830][52866] Updated weights for policy 1, policy_version 30630 (0.0007) -[2023-10-15 16:01:13,072][52833] Updated weights for policy 0, policy_version 30560 (0.0009) -[2023-10-15 16:01:13,195][52866] Updated weights for policy 1, policy_version 30640 (0.0008) -[2023-10-15 16:01:13,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 62652416. Throughput: 0: 1801.4, 1: 1800.5. Samples: 15674358. Policy #0 lag: (min: 34.0, avg: 47.3, max: 48.0) -[2023-10-15 16:01:13,442][51532] Avg episode reward: [(0, '34.900'), (1, '39.200')] -[2023-10-15 16:01:13,560][52866] Updated weights for policy 1, policy_version 30650 (0.0008) -[2023-10-15 16:01:16,809][52833] Updated weights for policy 0, policy_version 30570 (0.0009) -[2023-10-15 16:01:17,173][52833] Updated weights for policy 0, policy_version 30580 (0.0009) -[2023-10-15 16:01:17,314][52866] Updated weights for policy 1, policy_version 30660 (0.0008) -[2023-10-15 16:01:17,548][52833] Updated weights for policy 0, policy_version 30590 (0.0008) -[2023-10-15 16:01:17,670][52866] Updated weights for policy 1, policy_version 30670 (0.0007) -[2023-10-15 16:01:18,035][52866] Updated weights for policy 1, policy_version 30680 (0.0008) -[2023-10-15 16:01:18,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 62750720. Throughput: 0: 1798.4, 1: 1775.3. Samples: 15685700. Policy #0 lag: (min: 34.0, avg: 47.3, max: 48.0) -[2023-10-15 16:01:18,442][51532] Avg episode reward: [(0, '34.310'), (1, '41.310')] -[2023-10-15 16:01:21,285][52833] Updated weights for policy 0, policy_version 30600 (0.0008) -[2023-10-15 16:01:21,653][52833] Updated weights for policy 0, policy_version 30610 (0.0009) -[2023-10-15 16:01:21,784][52866] Updated weights for policy 1, policy_version 30690 (0.0007) -[2023-10-15 16:01:22,030][52833] Updated weights for policy 0, policy_version 30620 (0.0007) -[2023-10-15 16:01:22,153][52866] Updated weights for policy 1, policy_version 30700 (0.0008) -[2023-10-15 16:01:22,535][52866] Updated weights for policy 1, policy_version 30710 (0.0008) -[2023-10-15 16:01:22,898][52866] Updated weights for policy 1, policy_version 30720 (0.0008) -[2023-10-15 16:01:23,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62816256. Throughput: 0: 1807.2, 1: 1800.2. Samples: 15706848. Policy #0 lag: (min: 34.0, avg: 47.3, max: 48.0) -[2023-10-15 16:01:23,442][51532] Avg episode reward: [(0, '35.390'), (1, '39.420')] -[2023-10-15 16:01:25,839][52833] Updated weights for policy 0, policy_version 30630 (0.0007) -[2023-10-15 16:01:26,205][52833] Updated weights for policy 0, policy_version 30640 (0.0009) -[2023-10-15 16:01:26,577][52833] Updated weights for policy 0, policy_version 30650 (0.0007) -[2023-10-15 16:01:26,609][52866] Updated weights for policy 1, policy_version 30730 (0.0008) -[2023-10-15 16:01:26,976][52866] Updated weights for policy 1, policy_version 30740 (0.0008) -[2023-10-15 16:01:27,345][52866] Updated weights for policy 1, policy_version 30750 (0.0011) -[2023-10-15 16:01:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 62881792. Throughput: 0: 1790.8, 1: 1778.9. Samples: 15727618. Policy #0 lag: (min: 34.0, avg: 47.3, max: 48.0) -[2023-10-15 16:01:28,442][51532] Avg episode reward: [(0, '34.880'), (1, '38.130')] -[2023-10-15 16:01:30,317][52833] Updated weights for policy 0, policy_version 30660 (0.0009) -[2023-10-15 16:01:30,686][52833] Updated weights for policy 0, policy_version 30670 (0.0011) -[2023-10-15 16:01:31,061][52833] Updated weights for policy 0, policy_version 30680 (0.0008) -[2023-10-15 16:01:31,224][52866] Updated weights for policy 1, policy_version 30760 (0.0009) -[2023-10-15 16:01:31,588][52866] Updated weights for policy 1, policy_version 30770 (0.0008) -[2023-10-15 16:01:31,957][52866] Updated weights for policy 1, policy_version 30780 (0.0010) -[2023-10-15 16:01:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62947328. Throughput: 0: 1807.3, 1: 1800.0. Samples: 15739372. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) -[2023-10-15 16:01:33,442][51532] Avg episode reward: [(0, '36.360'), (1, '38.300')] -[2023-10-15 16:01:34,814][52833] Updated weights for policy 0, policy_version 30690 (0.0008) -[2023-10-15 16:01:35,187][52833] Updated weights for policy 0, policy_version 30700 (0.0007) -[2023-10-15 16:01:35,553][52833] Updated weights for policy 0, policy_version 30710 (0.0009) -[2023-10-15 16:01:35,761][52866] Updated weights for policy 1, policy_version 30790 (0.0009) -[2023-10-15 16:01:35,931][52833] Updated weights for policy 0, policy_version 30720 (0.0008) -[2023-10-15 16:01:36,130][52866] Updated weights for policy 1, policy_version 30800 (0.0008) -[2023-10-15 16:01:36,514][52866] Updated weights for policy 1, policy_version 30810 (0.0010) -[2023-10-15 16:01:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 63012864. Throughput: 0: 1793.7, 1: 1778.3. Samples: 15759872. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) -[2023-10-15 16:01:38,442][51532] Avg episode reward: [(0, '37.450'), (1, '38.540')] -[2023-10-15 16:01:39,622][52833] Updated weights for policy 0, policy_version 30730 (0.0008) -[2023-10-15 16:01:39,991][52833] Updated weights for policy 0, policy_version 30740 (0.0010) -[2023-10-15 16:01:40,333][52866] Updated weights for policy 1, policy_version 30820 (0.0009) -[2023-10-15 16:01:40,357][52833] Updated weights for policy 0, policy_version 30750 (0.0008) -[2023-10-15 16:01:40,710][52866] Updated weights for policy 1, policy_version 30830 (0.0008) -[2023-10-15 16:01:41,079][52866] Updated weights for policy 1, policy_version 30840 (0.0007) -[2023-10-15 16:01:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 63078400. Throughput: 0: 1795.5, 1: 1774.4. Samples: 15782184. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) -[2023-10-15 16:01:43,442][51532] Avg episode reward: [(0, '36.680'), (1, '34.700')] -[2023-10-15 16:01:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000030848_31588352.pth... -[2023-10-15 16:01:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000030752_31490048.pth... -[2023-10-15 16:01:43,501][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000029184_29884416.pth -[2023-10-15 16:01:43,501][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000029088_29786112.pth -[2023-10-15 16:01:44,140][52833] Updated weights for policy 0, policy_version 30760 (0.0008) -[2023-10-15 16:01:44,509][52833] Updated weights for policy 0, policy_version 30770 (0.0009) -[2023-10-15 16:01:44,841][52866] Updated weights for policy 1, policy_version 30850 (0.0008) -[2023-10-15 16:01:44,886][52833] Updated weights for policy 0, policy_version 30780 (0.0007) -[2023-10-15 16:01:45,212][52866] Updated weights for policy 1, policy_version 30860 (0.0010) -[2023-10-15 16:01:45,583][52866] Updated weights for policy 1, policy_version 30870 (0.0010) -[2023-10-15 16:01:45,946][52866] Updated weights for policy 1, policy_version 30880 (0.0008) -[2023-10-15 16:01:48,442][51532] Fps is (10 sec: 13106.1, 60 sec: 14199.2, 300 sec: 14329.0). Total num frames: 63143936. Throughput: 0: 1794.0, 1: 1776.5. Samples: 15792086. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) -[2023-10-15 16:01:48,443][51532] Avg episode reward: [(0, '37.690'), (1, '33.870')] -[2023-10-15 16:01:48,691][52833] Updated weights for policy 0, policy_version 30790 (0.0008) -[2023-10-15 16:01:49,066][52833] Updated weights for policy 0, policy_version 30800 (0.0008) -[2023-10-15 16:01:49,437][52833] Updated weights for policy 0, policy_version 30810 (0.0009) -[2023-10-15 16:01:49,731][52866] Updated weights for policy 1, policy_version 30890 (0.0009) -[2023-10-15 16:01:50,105][52866] Updated weights for policy 1, policy_version 30900 (0.0008) -[2023-10-15 16:01:50,470][52866] Updated weights for policy 1, policy_version 30910 (0.0007) -[2023-10-15 16:01:53,197][52833] Updated weights for policy 0, policy_version 30820 (0.0007) -[2023-10-15 16:01:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 63209472. Throughput: 0: 1795.0, 1: 1786.8. Samples: 15814606. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) -[2023-10-15 16:01:53,442][51532] Avg episode reward: [(0, '36.700'), (1, '33.520')] -[2023-10-15 16:01:53,574][52833] Updated weights for policy 0, policy_version 30830 (0.0007) -[2023-10-15 16:01:53,936][52833] Updated weights for policy 0, policy_version 30840 (0.0009) -[2023-10-15 16:01:54,210][52866] Updated weights for policy 1, policy_version 30920 (0.0007) -[2023-10-15 16:01:54,578][52866] Updated weights for policy 1, policy_version 30930 (0.0008) -[2023-10-15 16:01:54,949][52866] Updated weights for policy 1, policy_version 30940 (0.0007) -[2023-10-15 16:01:57,902][52833] Updated weights for policy 0, policy_version 30850 (0.0007) -[2023-10-15 16:01:58,304][52833] Updated weights for policy 0, policy_version 30860 (0.0009) -[2023-10-15 16:01:58,441][51532] Fps is (10 sec: 13108.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 63275008. Throughput: 0: 1817.2, 1: 1791.8. Samples: 15836764. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) -[2023-10-15 16:01:58,442][51532] Avg episode reward: [(0, '39.300'), (1, '34.340')] -[2023-10-15 16:01:58,674][52833] Updated weights for policy 0, policy_version 30870 (0.0009) -[2023-10-15 16:01:58,677][52866] Updated weights for policy 1, policy_version 30950 (0.0007) -[2023-10-15 16:01:59,045][52833] Updated weights for policy 0, policy_version 30880 (0.0008) -[2023-10-15 16:01:59,054][52866] Updated weights for policy 1, policy_version 30960 (0.0010) -[2023-10-15 16:01:59,417][52866] Updated weights for policy 1, policy_version 30970 (0.0008) -[2023-10-15 16:02:02,756][52833] Updated weights for policy 0, policy_version 30890 (0.0008) -[2023-10-15 16:02:03,127][52833] Updated weights for policy 0, policy_version 30900 (0.0007) -[2023-10-15 16:02:03,137][52866] Updated weights for policy 1, policy_version 30980 (0.0008) -[2023-10-15 16:02:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 63340544. Throughput: 0: 1789.5, 1: 1785.7. Samples: 15846582. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) -[2023-10-15 16:02:03,441][51532] Avg episode reward: [(0, '37.890'), (1, '35.060')] -[2023-10-15 16:02:03,497][52833] Updated weights for policy 0, policy_version 30910 (0.0007) -[2023-10-15 16:02:03,506][52866] Updated weights for policy 1, policy_version 30990 (0.0009) -[2023-10-15 16:02:03,872][52866] Updated weights for policy 1, policy_version 31000 (0.0008) -[2023-10-15 16:02:07,108][52833] Updated weights for policy 0, policy_version 30920 (0.0007) -[2023-10-15 16:02:07,484][52833] Updated weights for policy 0, policy_version 30930 (0.0008) -[2023-10-15 16:02:07,582][52866] Updated weights for policy 1, policy_version 31010 (0.0007) -[2023-10-15 16:02:07,854][52833] Updated weights for policy 0, policy_version 30940 (0.0009) -[2023-10-15 16:02:07,938][52866] Updated weights for policy 1, policy_version 31020 (0.0007) -[2023-10-15 16:02:08,298][52866] Updated weights for policy 1, policy_version 31030 (0.0007) -[2023-10-15 16:02:08,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 63438848. Throughput: 0: 1813.7, 1: 1790.1. Samples: 15869020. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) -[2023-10-15 16:02:08,441][51532] Avg episode reward: [(0, '37.320'), (1, '37.480')] -[2023-10-15 16:02:08,665][52866] Updated weights for policy 1, policy_version 31040 (0.0010) -[2023-10-15 16:02:11,529][52833] Updated weights for policy 0, policy_version 30950 (0.0008) -[2023-10-15 16:02:11,896][52833] Updated weights for policy 0, policy_version 30960 (0.0008) -[2023-10-15 16:02:12,274][52833] Updated weights for policy 0, policy_version 30970 (0.0008) -[2023-10-15 16:02:12,330][52866] Updated weights for policy 1, policy_version 31050 (0.0008) -[2023-10-15 16:02:12,692][52866] Updated weights for policy 1, policy_version 31060 (0.0009) -[2023-10-15 16:02:13,055][52866] Updated weights for policy 1, policy_version 31070 (0.0010) -[2023-10-15 16:02:13,441][51532] Fps is (10 sec: 19660.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 63537152. Throughput: 0: 1798.2, 1: 1793.8. Samples: 15889256. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) -[2023-10-15 16:02:13,442][51532] Avg episode reward: [(0, '35.130'), (1, '38.800')] -[2023-10-15 16:02:16,028][52833] Updated weights for policy 0, policy_version 30980 (0.0008) -[2023-10-15 16:02:16,401][52833] Updated weights for policy 0, policy_version 30990 (0.0011) -[2023-10-15 16:02:16,773][52833] Updated weights for policy 0, policy_version 31000 (0.0009) -[2023-10-15 16:02:16,887][52866] Updated weights for policy 1, policy_version 31080 (0.0010) -[2023-10-15 16:02:17,252][52866] Updated weights for policy 1, policy_version 31090 (0.0007) -[2023-10-15 16:02:17,618][52866] Updated weights for policy 1, policy_version 31100 (0.0009) -[2023-10-15 16:02:18,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63602688. Throughput: 0: 1811.8, 1: 1795.1. Samples: 15901684. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-15 16:02:18,442][51532] Avg episode reward: [(0, '35.130'), (1, '39.620')] -[2023-10-15 16:02:20,510][52833] Updated weights for policy 0, policy_version 31010 (0.0008) -[2023-10-15 16:02:20,888][52833] Updated weights for policy 0, policy_version 31020 (0.0008) -[2023-10-15 16:02:21,247][52833] Updated weights for policy 0, policy_version 31030 (0.0009) -[2023-10-15 16:02:21,420][52866] Updated weights for policy 1, policy_version 31110 (0.0010) -[2023-10-15 16:02:21,619][52833] Updated weights for policy 0, policy_version 31040 (0.0008) -[2023-10-15 16:02:21,794][52866] Updated weights for policy 1, policy_version 31120 (0.0009) -[2023-10-15 16:02:22,155][52866] Updated weights for policy 1, policy_version 31130 (0.0010) -[2023-10-15 16:02:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63668224. Throughput: 0: 1789.2, 1: 1806.7. Samples: 15921688. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-15 16:02:23,442][51532] Avg episode reward: [(0, '34.690'), (1, '39.660')] -[2023-10-15 16:02:25,184][52833] Updated weights for policy 0, policy_version 31050 (0.0009) -[2023-10-15 16:02:25,557][52833] Updated weights for policy 0, policy_version 31060 (0.0011) -[2023-10-15 16:02:25,915][52866] Updated weights for policy 1, policy_version 31140 (0.0009) -[2023-10-15 16:02:25,924][52833] Updated weights for policy 0, policy_version 31070 (0.0008) -[2023-10-15 16:02:26,284][52866] Updated weights for policy 1, policy_version 31150 (0.0008) -[2023-10-15 16:02:26,663][52866] Updated weights for policy 1, policy_version 31160 (0.0009) -[2023-10-15 16:02:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63733760. Throughput: 0: 1790.8, 1: 1798.0. Samples: 15943676. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-15 16:02:28,442][51532] Avg episode reward: [(0, '35.310'), (1, '40.270')] -[2023-10-15 16:02:29,618][52833] Updated weights for policy 0, policy_version 31080 (0.0009) -[2023-10-15 16:02:29,982][52833] Updated weights for policy 0, policy_version 31090 (0.0009) -[2023-10-15 16:02:30,347][52833] Updated weights for policy 0, policy_version 31100 (0.0008) -[2023-10-15 16:02:30,453][52866] Updated weights for policy 1, policy_version 31170 (0.0009) -[2023-10-15 16:02:30,819][52866] Updated weights for policy 1, policy_version 31180 (0.0011) -[2023-10-15 16:02:31,187][52866] Updated weights for policy 1, policy_version 31190 (0.0011) -[2023-10-15 16:02:31,557][52866] Updated weights for policy 1, policy_version 31200 (0.0010) -[2023-10-15 16:02:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 63799296. Throughput: 0: 1791.0, 1: 1813.8. Samples: 15954298. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-15 16:02:33,442][51532] Avg episode reward: [(0, '35.520'), (1, '44.280')] -[2023-10-15 16:02:33,443][52518] Saving new best policy, reward=44.280! -[2023-10-15 16:02:34,088][52833] Updated weights for policy 0, policy_version 31110 (0.0007) -[2023-10-15 16:02:34,452][52833] Updated weights for policy 0, policy_version 31120 (0.0009) -[2023-10-15 16:02:34,818][52833] Updated weights for policy 0, policy_version 31130 (0.0008) -[2023-10-15 16:02:35,260][52866] Updated weights for policy 1, policy_version 31210 (0.0010) -[2023-10-15 16:02:35,621][52866] Updated weights for policy 1, policy_version 31220 (0.0008) -[2023-10-15 16:02:35,984][52866] Updated weights for policy 1, policy_version 31230 (0.0010) -[2023-10-15 16:02:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 63864832. Throughput: 0: 1797.9, 1: 1793.5. Samples: 15976218. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-15 16:02:38,442][51532] Avg episode reward: [(0, '33.890'), (1, '42.670')] -[2023-10-15 16:02:38,492][52833] Updated weights for policy 0, policy_version 31140 (0.0007) -[2023-10-15 16:02:38,855][52833] Updated weights for policy 0, policy_version 31150 (0.0008) -[2023-10-15 16:02:39,230][52833] Updated weights for policy 0, policy_version 31160 (0.0007) -[2023-10-15 16:02:39,670][52866] Updated weights for policy 1, policy_version 31240 (0.0008) -[2023-10-15 16:02:40,046][52866] Updated weights for policy 1, policy_version 31250 (0.0008) -[2023-10-15 16:02:40,412][52866] Updated weights for policy 1, policy_version 31260 (0.0008) -[2023-10-15 16:02:43,133][52833] Updated weights for policy 0, policy_version 31170 (0.0009) -[2023-10-15 16:02:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 63930368. Throughput: 0: 1802.6, 1: 1794.7. Samples: 15998642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:02:43,442][51532] Avg episode reward: [(0, '35.090'), (1, '46.470')] -[2023-10-15 16:02:43,452][52518] Saving new best policy, reward=46.470! -[2023-10-15 16:02:43,536][52833] Updated weights for policy 0, policy_version 31180 (0.0007) -[2023-10-15 16:02:43,908][52833] Updated weights for policy 0, policy_version 31190 (0.0009) -[2023-10-15 16:02:44,268][52833] Updated weights for policy 0, policy_version 31200 (0.0007) -[2023-10-15 16:02:44,278][52866] Updated weights for policy 1, policy_version 31270 (0.0009) -[2023-10-15 16:02:44,642][52866] Updated weights for policy 1, policy_version 31280 (0.0008) -[2023-10-15 16:02:45,010][52866] Updated weights for policy 1, policy_version 31290 (0.0008) -[2023-10-15 16:02:47,919][52833] Updated weights for policy 0, policy_version 31210 (0.0008) -[2023-10-15 16:02:48,285][52833] Updated weights for policy 0, policy_version 31220 (0.0007) -[2023-10-15 16:02:48,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.7, 300 sec: 14218.0). Total num frames: 63995904. Throughput: 0: 1800.1, 1: 1797.1. Samples: 16008456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:02:48,441][51532] Avg episode reward: [(0, '35.150'), (1, '44.290')] -[2023-10-15 16:02:48,663][52833] Updated weights for policy 0, policy_version 31230 (0.0009) -[2023-10-15 16:02:48,846][52866] Updated weights for policy 1, policy_version 31300 (0.0007) -[2023-10-15 16:02:49,212][52866] Updated weights for policy 1, policy_version 31310 (0.0009) -[2023-10-15 16:02:49,580][52866] Updated weights for policy 1, policy_version 31320 (0.0008) -[2023-10-15 16:02:52,380][52833] Updated weights for policy 0, policy_version 31240 (0.0007) -[2023-10-15 16:02:52,743][52833] Updated weights for policy 0, policy_version 31250 (0.0008) -[2023-10-15 16:02:53,114][52833] Updated weights for policy 0, policy_version 31260 (0.0007) -[2023-10-15 16:02:53,203][52866] Updated weights for policy 1, policy_version 31330 (0.0007) -[2023-10-15 16:02:53,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 64094208. Throughput: 0: 1804.8, 1: 1796.0. Samples: 16031054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:02:53,442][51532] Avg episode reward: [(0, '35.740'), (1, '42.790')] -[2023-10-15 16:02:53,571][52866] Updated weights for policy 1, policy_version 31340 (0.0008) -[2023-10-15 16:02:53,947][52866] Updated weights for policy 1, policy_version 31350 (0.0007) -[2023-10-15 16:02:54,306][52866] Updated weights for policy 1, policy_version 31360 (0.0007) -[2023-10-15 16:02:56,930][52833] Updated weights for policy 0, policy_version 31270 (0.0007) -[2023-10-15 16:02:57,301][52833] Updated weights for policy 0, policy_version 31280 (0.0008) -[2023-10-15 16:02:57,671][52833] Updated weights for policy 0, policy_version 31290 (0.0009) -[2023-10-15 16:02:58,015][52866] Updated weights for policy 1, policy_version 31370 (0.0009) -[2023-10-15 16:02:58,386][52866] Updated weights for policy 1, policy_version 31380 (0.0008) -[2023-10-15 16:02:58,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 64159744. Throughput: 0: 1801.7, 1: 1814.4. Samples: 16051980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:02:58,442][51532] Avg episode reward: [(0, '36.390'), (1, '39.990')] -[2023-10-15 16:02:58,750][52866] Updated weights for policy 1, policy_version 31390 (0.0008) -[2023-10-15 16:03:01,406][52833] Updated weights for policy 0, policy_version 31300 (0.0008) -[2023-10-15 16:03:01,771][52833] Updated weights for policy 0, policy_version 31310 (0.0007) -[2023-10-15 16:03:02,147][52833] Updated weights for policy 0, policy_version 31320 (0.0007) -[2023-10-15 16:03:02,401][52866] Updated weights for policy 1, policy_version 31400 (0.0007) -[2023-10-15 16:03:02,779][52866] Updated weights for policy 1, policy_version 31410 (0.0009) -[2023-10-15 16:03:03,157][52866] Updated weights for policy 1, policy_version 31420 (0.0010) -[2023-10-15 16:03:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 64258048. Throughput: 0: 1802.9, 1: 1796.3. Samples: 16063646. Policy #0 lag: (min: 10.0, avg: 29.9, max: 32.0) -[2023-10-15 16:03:03,441][51532] Avg episode reward: [(0, '34.050'), (1, '39.060')] -[2023-10-15 16:03:05,772][52833] Updated weights for policy 0, policy_version 31330 (0.0008) -[2023-10-15 16:03:06,137][52833] Updated weights for policy 0, policy_version 31340 (0.0009) -[2023-10-15 16:03:06,512][52833] Updated weights for policy 0, policy_version 31350 (0.0008) -[2023-10-15 16:03:06,882][52833] Updated weights for policy 0, policy_version 31360 (0.0007) -[2023-10-15 16:03:06,970][52866] Updated weights for policy 1, policy_version 31430 (0.0008) -[2023-10-15 16:03:07,343][52866] Updated weights for policy 1, policy_version 31440 (0.0009) -[2023-10-15 16:03:07,719][52866] Updated weights for policy 1, policy_version 31450 (0.0009) -[2023-10-15 16:03:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 64323584. Throughput: 0: 1814.0, 1: 1803.2. Samples: 16084460. Policy #0 lag: (min: 10.0, avg: 29.9, max: 32.0) -[2023-10-15 16:03:08,442][51532] Avg episode reward: [(0, '34.080'), (1, '38.180')] -[2023-10-15 16:03:10,402][52833] Updated weights for policy 0, policy_version 31370 (0.0009) -[2023-10-15 16:03:10,767][52833] Updated weights for policy 0, policy_version 31380 (0.0009) -[2023-10-15 16:03:11,132][52833] Updated weights for policy 0, policy_version 31390 (0.0008) -[2023-10-15 16:03:11,589][52866] Updated weights for policy 1, policy_version 31460 (0.0007) -[2023-10-15 16:03:11,956][52866] Updated weights for policy 1, policy_version 31470 (0.0008) -[2023-10-15 16:03:12,323][52866] Updated weights for policy 1, policy_version 31480 (0.0008) -[2023-10-15 16:03:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 64389120. Throughput: 0: 1818.5, 1: 1792.0. Samples: 16106152. Policy #0 lag: (min: 10.0, avg: 29.9, max: 32.0) -[2023-10-15 16:03:13,442][51532] Avg episode reward: [(0, '34.330'), (1, '35.980')] -[2023-10-15 16:03:14,864][52833] Updated weights for policy 0, policy_version 31400 (0.0008) -[2023-10-15 16:03:15,229][52833] Updated weights for policy 0, policy_version 31410 (0.0010) -[2023-10-15 16:03:15,596][52833] Updated weights for policy 0, policy_version 31420 (0.0009) -[2023-10-15 16:03:16,008][52866] Updated weights for policy 1, policy_version 31490 (0.0008) -[2023-10-15 16:03:16,379][52866] Updated weights for policy 1, policy_version 31500 (0.0011) -[2023-10-15 16:03:16,737][52866] Updated weights for policy 1, policy_version 31510 (0.0009) -[2023-10-15 16:03:17,101][52866] Updated weights for policy 1, policy_version 31520 (0.0008) -[2023-10-15 16:03:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 64454656. Throughput: 0: 1814.2, 1: 1808.1. Samples: 16117304. Policy #0 lag: (min: 10.0, avg: 29.9, max: 32.0) -[2023-10-15 16:03:18,441][51532] Avg episode reward: [(0, '33.460'), (1, '37.080')] -[2023-10-15 16:03:19,431][52833] Updated weights for policy 0, policy_version 31430 (0.0008) -[2023-10-15 16:03:19,804][52833] Updated weights for policy 0, policy_version 31440 (0.0010) -[2023-10-15 16:03:20,175][52833] Updated weights for policy 0, policy_version 31450 (0.0009) -[2023-10-15 16:03:20,873][52866] Updated weights for policy 1, policy_version 31530 (0.0007) -[2023-10-15 16:03:21,247][52866] Updated weights for policy 1, policy_version 31540 (0.0009) -[2023-10-15 16:03:21,605][52866] Updated weights for policy 1, policy_version 31550 (0.0009) -[2023-10-15 16:03:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 64520192. Throughput: 0: 1811.8, 1: 1792.6. Samples: 16138416. Policy #0 lag: (min: 10.0, avg: 29.9, max: 32.0) -[2023-10-15 16:03:23,442][51532] Avg episode reward: [(0, '36.300'), (1, '37.320')] -[2023-10-15 16:03:23,703][52833] Updated weights for policy 0, policy_version 31460 (0.0010) -[2023-10-15 16:03:24,073][52833] Updated weights for policy 0, policy_version 31470 (0.0009) -[2023-10-15 16:03:24,443][52833] Updated weights for policy 0, policy_version 31480 (0.0011) -[2023-10-15 16:03:25,303][52866] Updated weights for policy 1, policy_version 31560 (0.0010) -[2023-10-15 16:03:25,678][52866] Updated weights for policy 1, policy_version 31570 (0.0011) -[2023-10-15 16:03:26,044][52866] Updated weights for policy 1, policy_version 31580 (0.0009) -[2023-10-15 16:03:28,298][52833] Updated weights for policy 0, policy_version 31490 (0.0008) -[2023-10-15 16:03:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 64585728. Throughput: 0: 1812.9, 1: 1796.9. Samples: 16161082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:03:28,441][51532] Avg episode reward: [(0, '36.790'), (1, '38.220')] -[2023-10-15 16:03:28,709][52833] Updated weights for policy 0, policy_version 31500 (0.0010) -[2023-10-15 16:03:29,076][52833] Updated weights for policy 0, policy_version 31510 (0.0007) -[2023-10-15 16:03:29,453][52833] Updated weights for policy 0, policy_version 31520 (0.0007) -[2023-10-15 16:03:29,832][52866] Updated weights for policy 1, policy_version 31590 (0.0008) -[2023-10-15 16:03:30,197][52866] Updated weights for policy 1, policy_version 31600 (0.0008) -[2023-10-15 16:03:30,560][52866] Updated weights for policy 1, policy_version 31610 (0.0009) -[2023-10-15 16:03:32,911][52833] Updated weights for policy 0, policy_version 31530 (0.0010) -[2023-10-15 16:03:33,286][52833] Updated weights for policy 0, policy_version 31540 (0.0011) -[2023-10-15 16:03:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 64651264. Throughput: 0: 1814.3, 1: 1795.5. Samples: 16170898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:03:33,442][51532] Avg episode reward: [(0, '35.230'), (1, '38.440')] -[2023-10-15 16:03:33,650][52833] Updated weights for policy 0, policy_version 31550 (0.0009) -[2023-10-15 16:03:34,269][52866] Updated weights for policy 1, policy_version 31620 (0.0008) -[2023-10-15 16:03:34,634][52866] Updated weights for policy 1, policy_version 31630 (0.0011) -[2023-10-15 16:03:35,003][52866] Updated weights for policy 1, policy_version 31640 (0.0010) -[2023-10-15 16:03:37,644][52833] Updated weights for policy 0, policy_version 31560 (0.0010) -[2023-10-15 16:03:38,014][52833] Updated weights for policy 0, policy_version 31570 (0.0011) -[2023-10-15 16:03:38,390][52833] Updated weights for policy 0, policy_version 31580 (0.0009) -[2023-10-15 16:03:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 64716800. Throughput: 0: 1806.3, 1: 1798.7. Samples: 16193278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:03:38,441][51532] Avg episode reward: [(0, '36.400'), (1, '40.590')] -[2023-10-15 16:03:38,672][52866] Updated weights for policy 1, policy_version 31650 (0.0008) -[2023-10-15 16:03:39,031][52866] Updated weights for policy 1, policy_version 31660 (0.0007) -[2023-10-15 16:03:39,402][52866] Updated weights for policy 1, policy_version 31670 (0.0009) -[2023-10-15 16:03:39,768][52866] Updated weights for policy 1, policy_version 31680 (0.0007) -[2023-10-15 16:03:42,124][52833] Updated weights for policy 0, policy_version 31590 (0.0007) -[2023-10-15 16:03:42,491][52833] Updated weights for policy 0, policy_version 31600 (0.0008) -[2023-10-15 16:03:42,861][52833] Updated weights for policy 0, policy_version 31610 (0.0009) -[2023-10-15 16:03:43,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 64815104. Throughput: 0: 1810.3, 1: 1802.0. Samples: 16214536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:03:43,441][51532] Avg episode reward: [(0, '37.330'), (1, '41.130')] -[2023-10-15 16:03:43,448][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000031616_32374784.pth... -[2023-10-15 16:03:43,477][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000029920_30638080.pth -[2023-10-15 16:03:43,508][52866] Updated weights for policy 1, policy_version 31690 (0.0009) -[2023-10-15 16:03:43,870][52866] Updated weights for policy 1, policy_version 31700 (0.0007) -[2023-10-15 16:03:44,242][52866] Updated weights for policy 1, policy_version 31710 (0.0010) -[2023-10-15 16:03:44,309][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000031712_32473088.pth... -[2023-10-15 16:03:44,338][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000030016_30736384.pth -[2023-10-15 16:03:46,697][52833] Updated weights for policy 0, policy_version 31620 (0.0009) -[2023-10-15 16:03:47,070][52833] Updated weights for policy 0, policy_version 31630 (0.0010) -[2023-10-15 16:03:47,436][52833] Updated weights for policy 0, policy_version 31640 (0.0009) -[2023-10-15 16:03:47,943][52866] Updated weights for policy 1, policy_version 31720 (0.0008) -[2023-10-15 16:03:48,310][52866] Updated weights for policy 1, policy_version 31730 (0.0009) -[2023-10-15 16:03:48,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 64880640. Throughput: 0: 1806.8, 1: 1787.0. Samples: 16225364. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 16:03:48,441][51532] Avg episode reward: [(0, '37.440'), (1, '41.430')] -[2023-10-15 16:03:48,673][52866] Updated weights for policy 1, policy_version 31740 (0.0010) -[2023-10-15 16:03:51,127][52833] Updated weights for policy 0, policy_version 31650 (0.0009) -[2023-10-15 16:03:51,502][52833] Updated weights for policy 0, policy_version 31660 (0.0009) -[2023-10-15 16:03:51,870][52833] Updated weights for policy 0, policy_version 31670 (0.0009) -[2023-10-15 16:03:52,240][52833] Updated weights for policy 0, policy_version 31680 (0.0009) -[2023-10-15 16:03:52,391][52866] Updated weights for policy 1, policy_version 31750 (0.0008) -[2023-10-15 16:03:52,754][52866] Updated weights for policy 1, policy_version 31760 (0.0008) -[2023-10-15 16:03:53,128][52866] Updated weights for policy 1, policy_version 31770 (0.0010) -[2023-10-15 16:03:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 64978944. Throughput: 0: 1812.2, 1: 1799.5. Samples: 16246988. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 16:03:53,442][51532] Avg episode reward: [(0, '36.240'), (1, '41.660')] -[2023-10-15 16:03:55,956][52833] Updated weights for policy 0, policy_version 31690 (0.0010) -[2023-10-15 16:03:56,316][52833] Updated weights for policy 0, policy_version 31700 (0.0009) -[2023-10-15 16:03:56,685][52833] Updated weights for policy 0, policy_version 31710 (0.0010) -[2023-10-15 16:03:56,897][52866] Updated weights for policy 1, policy_version 31780 (0.0009) -[2023-10-15 16:03:57,265][52866] Updated weights for policy 1, policy_version 31790 (0.0009) -[2023-10-15 16:03:57,625][52866] Updated weights for policy 1, policy_version 31800 (0.0009) -[2023-10-15 16:03:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 65044480. Throughput: 0: 1789.6, 1: 1792.1. Samples: 16267326. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 16:03:58,441][51532] Avg episode reward: [(0, '37.970'), (1, '42.090')] -[2023-10-15 16:04:00,455][52833] Updated weights for policy 0, policy_version 31720 (0.0008) -[2023-10-15 16:04:00,830][52833] Updated weights for policy 0, policy_version 31730 (0.0009) -[2023-10-15 16:04:01,192][52833] Updated weights for policy 0, policy_version 31740 (0.0008) -[2023-10-15 16:04:01,306][52866] Updated weights for policy 1, policy_version 31810 (0.0010) -[2023-10-15 16:04:01,666][52866] Updated weights for policy 1, policy_version 31820 (0.0009) -[2023-10-15 16:04:02,041][52866] Updated weights for policy 1, policy_version 31830 (0.0008) -[2023-10-15 16:04:02,407][52866] Updated weights for policy 1, policy_version 31840 (0.0008) -[2023-10-15 16:04:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65110016. Throughput: 0: 1804.7, 1: 1796.8. Samples: 16279372. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 16:04:03,442][51532] Avg episode reward: [(0, '39.100'), (1, '41.570')] -[2023-10-15 16:04:04,842][52833] Updated weights for policy 0, policy_version 31750 (0.0009) -[2023-10-15 16:04:05,213][52833] Updated weights for policy 0, policy_version 31760 (0.0007) -[2023-10-15 16:04:05,575][52833] Updated weights for policy 0, policy_version 31770 (0.0007) -[2023-10-15 16:04:06,278][52866] Updated weights for policy 1, policy_version 31850 (0.0010) -[2023-10-15 16:04:06,647][52866] Updated weights for policy 1, policy_version 31860 (0.0009) -[2023-10-15 16:04:07,011][52866] Updated weights for policy 1, policy_version 31870 (0.0007) -[2023-10-15 16:04:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65175552. Throughput: 0: 1789.5, 1: 1803.0. Samples: 16300078. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 16:04:08,442][51532] Avg episode reward: [(0, '38.720'), (1, '40.890')] -[2023-10-15 16:04:09,382][52833] Updated weights for policy 0, policy_version 31780 (0.0008) -[2023-10-15 16:04:09,759][52833] Updated weights for policy 0, policy_version 31790 (0.0010) -[2023-10-15 16:04:10,124][52833] Updated weights for policy 0, policy_version 31800 (0.0010) -[2023-10-15 16:04:10,695][52866] Updated weights for policy 1, policy_version 31880 (0.0008) -[2023-10-15 16:04:11,066][52866] Updated weights for policy 1, policy_version 31890 (0.0010) -[2023-10-15 16:04:11,435][52866] Updated weights for policy 1, policy_version 31900 (0.0012) -[2023-10-15 16:04:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65241088. Throughput: 0: 1785.9, 1: 1798.4. Samples: 16322374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:04:13,441][51532] Avg episode reward: [(0, '38.030'), (1, '39.550')] -[2023-10-15 16:04:14,026][52833] Updated weights for policy 0, policy_version 31810 (0.0009) -[2023-10-15 16:04:14,395][52833] Updated weights for policy 0, policy_version 31820 (0.0008) -[2023-10-15 16:04:14,768][52833] Updated weights for policy 0, policy_version 31830 (0.0008) -[2023-10-15 16:04:15,142][52833] Updated weights for policy 0, policy_version 31840 (0.0007) -[2023-10-15 16:04:15,250][52866] Updated weights for policy 1, policy_version 31910 (0.0009) -[2023-10-15 16:04:15,609][52866] Updated weights for policy 1, policy_version 31920 (0.0010) -[2023-10-15 16:04:15,977][52866] Updated weights for policy 1, policy_version 31930 (0.0010) -[2023-10-15 16:04:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 65306624. Throughput: 0: 1787.3, 1: 1803.9. Samples: 16332498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:04:18,441][51532] Avg episode reward: [(0, '35.830'), (1, '39.160')] -[2023-10-15 16:04:18,775][52833] Updated weights for policy 0, policy_version 31850 (0.0008) -[2023-10-15 16:04:19,143][52833] Updated weights for policy 0, policy_version 31860 (0.0009) -[2023-10-15 16:04:19,518][52833] Updated weights for policy 0, policy_version 31870 (0.0009) -[2023-10-15 16:04:19,790][52866] Updated weights for policy 1, policy_version 31940 (0.0008) -[2023-10-15 16:04:20,160][52866] Updated weights for policy 1, policy_version 31950 (0.0011) -[2023-10-15 16:04:20,523][52866] Updated weights for policy 1, policy_version 31960 (0.0009) -[2023-10-15 16:04:23,330][52833] Updated weights for policy 0, policy_version 31880 (0.0009) -[2023-10-15 16:04:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 65372160. Throughput: 0: 1791.7, 1: 1792.7. Samples: 16354580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:04:23,442][51532] Avg episode reward: [(0, '35.190'), (1, '38.200')] -[2023-10-15 16:04:23,699][52833] Updated weights for policy 0, policy_version 31890 (0.0010) -[2023-10-15 16:04:24,071][52833] Updated weights for policy 0, policy_version 31900 (0.0010) -[2023-10-15 16:04:24,243][52866] Updated weights for policy 1, policy_version 31970 (0.0008) -[2023-10-15 16:04:24,606][52866] Updated weights for policy 1, policy_version 31980 (0.0009) -[2023-10-15 16:04:24,970][52866] Updated weights for policy 1, policy_version 31990 (0.0011) -[2023-10-15 16:04:25,345][52866] Updated weights for policy 1, policy_version 32000 (0.0010) -[2023-10-15 16:04:27,749][52833] Updated weights for policy 0, policy_version 31910 (0.0007) -[2023-10-15 16:04:28,118][52833] Updated weights for policy 0, policy_version 31920 (0.0008) -[2023-10-15 16:04:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 65437696. Throughput: 0: 1811.6, 1: 1798.4. Samples: 16376986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:04:28,441][51532] Avg episode reward: [(0, '35.090'), (1, '36.280')] -[2023-10-15 16:04:28,482][52833] Updated weights for policy 0, policy_version 31930 (0.0008) -[2023-10-15 16:04:29,008][52866] Updated weights for policy 1, policy_version 32010 (0.0008) -[2023-10-15 16:04:29,380][52866] Updated weights for policy 1, policy_version 32020 (0.0008) -[2023-10-15 16:04:29,756][52866] Updated weights for policy 1, policy_version 32030 (0.0009) -[2023-10-15 16:04:32,208][52833] Updated weights for policy 0, policy_version 31940 (0.0007) -[2023-10-15 16:04:32,596][52833] Updated weights for policy 0, policy_version 31950 (0.0008) -[2023-10-15 16:04:32,971][52833] Updated weights for policy 0, policy_version 31960 (0.0009) -[2023-10-15 16:04:33,351][52866] Updated weights for policy 1, policy_version 32040 (0.0008) -[2023-10-15 16:04:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 65536000. Throughput: 0: 1796.2, 1: 1803.4. Samples: 16387344. Policy #0 lag: (min: 6.0, avg: 9.0, max: 38.0) -[2023-10-15 16:04:33,442][51532] Avg episode reward: [(0, '37.500'), (1, '36.860')] -[2023-10-15 16:04:33,718][52866] Updated weights for policy 1, policy_version 32050 (0.0011) -[2023-10-15 16:04:34,082][52866] Updated weights for policy 1, policy_version 32060 (0.0009) -[2023-10-15 16:04:36,875][52833] Updated weights for policy 0, policy_version 31970 (0.0009) -[2023-10-15 16:04:37,243][52833] Updated weights for policy 0, policy_version 31980 (0.0010) -[2023-10-15 16:04:37,611][52833] Updated weights for policy 0, policy_version 31990 (0.0011) -[2023-10-15 16:04:37,974][52833] Updated weights for policy 0, policy_version 32000 (0.0009) -[2023-10-15 16:04:38,018][52866] Updated weights for policy 1, policy_version 32070 (0.0007) -[2023-10-15 16:04:38,384][52866] Updated weights for policy 1, policy_version 32080 (0.0010) -[2023-10-15 16:04:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 65601536. Throughput: 0: 1804.3, 1: 1803.7. Samples: 16409344. Policy #0 lag: (min: 6.0, avg: 9.0, max: 38.0) -[2023-10-15 16:04:38,441][51532] Avg episode reward: [(0, '34.380'), (1, '37.630')] -[2023-10-15 16:04:38,755][52866] Updated weights for policy 1, policy_version 32090 (0.0009) -[2023-10-15 16:04:41,583][52833] Updated weights for policy 0, policy_version 32010 (0.0010) -[2023-10-15 16:04:41,965][52833] Updated weights for policy 0, policy_version 32020 (0.0011) -[2023-10-15 16:04:42,342][52833] Updated weights for policy 0, policy_version 32030 (0.0009) -[2023-10-15 16:04:42,624][52866] Updated weights for policy 1, policy_version 32100 (0.0009) -[2023-10-15 16:04:43,000][52866] Updated weights for policy 1, policy_version 32110 (0.0008) -[2023-10-15 16:04:43,358][52866] Updated weights for policy 1, policy_version 32120 (0.0008) -[2023-10-15 16:04:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 65667072. Throughput: 0: 1790.5, 1: 1823.8. Samples: 16429968. Policy #0 lag: (min: 6.0, avg: 9.0, max: 38.0) -[2023-10-15 16:04:43,442][51532] Avg episode reward: [(0, '33.350'), (1, '36.120')] -[2023-10-15 16:04:46,033][52833] Updated weights for policy 0, policy_version 32040 (0.0007) -[2023-10-15 16:04:46,402][52833] Updated weights for policy 0, policy_version 32050 (0.0007) -[2023-10-15 16:04:46,774][52833] Updated weights for policy 0, policy_version 32060 (0.0010) -[2023-10-15 16:04:47,031][52866] Updated weights for policy 1, policy_version 32130 (0.0008) -[2023-10-15 16:04:47,393][52866] Updated weights for policy 1, policy_version 32140 (0.0010) -[2023-10-15 16:04:47,763][52866] Updated weights for policy 1, policy_version 32150 (0.0007) -[2023-10-15 16:04:48,133][52866] Updated weights for policy 1, policy_version 32160 (0.0007) -[2023-10-15 16:04:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 65765376. Throughput: 0: 1809.8, 1: 1799.5. Samples: 16441790. Policy #0 lag: (min: 6.0, avg: 9.0, max: 38.0) -[2023-10-15 16:04:48,441][51532] Avg episode reward: [(0, '33.620'), (1, '38.200')] -[2023-10-15 16:04:50,454][52833] Updated weights for policy 0, policy_version 32070 (0.0009) -[2023-10-15 16:04:50,821][52833] Updated weights for policy 0, policy_version 32080 (0.0007) -[2023-10-15 16:04:51,190][52833] Updated weights for policy 0, policy_version 32090 (0.0009) -[2023-10-15 16:04:51,898][52866] Updated weights for policy 1, policy_version 32170 (0.0007) -[2023-10-15 16:04:52,269][52866] Updated weights for policy 1, policy_version 32180 (0.0007) -[2023-10-15 16:04:52,632][52866] Updated weights for policy 1, policy_version 32190 (0.0008) -[2023-10-15 16:04:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65830912. Throughput: 0: 1792.4, 1: 1818.0. Samples: 16462546. Policy #0 lag: (min: 6.0, avg: 9.0, max: 38.0) -[2023-10-15 16:04:53,442][51532] Avg episode reward: [(0, '35.110'), (1, '36.810')] -[2023-10-15 16:04:54,823][52833] Updated weights for policy 0, policy_version 32100 (0.0008) -[2023-10-15 16:04:55,197][52833] Updated weights for policy 0, policy_version 32110 (0.0010) -[2023-10-15 16:04:55,563][52833] Updated weights for policy 0, policy_version 32120 (0.0011) -[2023-10-15 16:04:56,461][52866] Updated weights for policy 1, policy_version 32200 (0.0009) -[2023-10-15 16:04:56,837][52866] Updated weights for policy 1, policy_version 32210 (0.0009) -[2023-10-15 16:04:57,209][52866] Updated weights for policy 1, policy_version 32220 (0.0010) -[2023-10-15 16:04:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65896448. Throughput: 0: 1799.0, 1: 1792.5. Samples: 16483992. Policy #0 lag: (min: 14.0, avg: 28.3, max: 46.0) -[2023-10-15 16:04:58,442][51532] Avg episode reward: [(0, '38.760'), (1, '38.850')] -[2023-10-15 16:04:59,518][52833] Updated weights for policy 0, policy_version 32130 (0.0010) -[2023-10-15 16:04:59,930][52833] Updated weights for policy 0, policy_version 32140 (0.0009) -[2023-10-15 16:05:00,305][52833] Updated weights for policy 0, policy_version 32150 (0.0011) -[2023-10-15 16:05:00,677][52833] Updated weights for policy 0, policy_version 32160 (0.0010) -[2023-10-15 16:05:00,921][52866] Updated weights for policy 1, policy_version 32230 (0.0008) -[2023-10-15 16:05:01,288][52866] Updated weights for policy 1, policy_version 32240 (0.0007) -[2023-10-15 16:05:01,657][52866] Updated weights for policy 1, policy_version 32250 (0.0008) -[2023-10-15 16:05:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 65961984. Throughput: 0: 1791.6, 1: 1812.5. Samples: 16494682. Policy #0 lag: (min: 14.0, avg: 28.3, max: 46.0) -[2023-10-15 16:05:03,442][51532] Avg episode reward: [(0, '39.130'), (1, '38.890')] -[2023-10-15 16:05:04,321][52833] Updated weights for policy 0, policy_version 32170 (0.0011) -[2023-10-15 16:05:04,683][52833] Updated weights for policy 0, policy_version 32180 (0.0008) -[2023-10-15 16:05:05,054][52833] Updated weights for policy 0, policy_version 32190 (0.0007) -[2023-10-15 16:05:05,246][52866] Updated weights for policy 1, policy_version 32260 (0.0008) -[2023-10-15 16:05:05,619][52866] Updated weights for policy 1, policy_version 32270 (0.0008) -[2023-10-15 16:05:05,978][52866] Updated weights for policy 1, policy_version 32280 (0.0007) -[2023-10-15 16:05:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66027520. Throughput: 0: 1791.9, 1: 1797.7. Samples: 16516112. Policy #0 lag: (min: 14.0, avg: 28.3, max: 46.0) -[2023-10-15 16:05:08,442][51532] Avg episode reward: [(0, '36.940'), (1, '37.840')] -[2023-10-15 16:05:08,886][52833] Updated weights for policy 0, policy_version 32200 (0.0009) -[2023-10-15 16:05:09,258][52833] Updated weights for policy 0, policy_version 32210 (0.0009) -[2023-10-15 16:05:09,636][52833] Updated weights for policy 0, policy_version 32220 (0.0011) -[2023-10-15 16:05:09,683][52866] Updated weights for policy 1, policy_version 32290 (0.0009) -[2023-10-15 16:05:10,059][52866] Updated weights for policy 1, policy_version 32300 (0.0010) -[2023-10-15 16:05:10,425][52866] Updated weights for policy 1, policy_version 32310 (0.0012) -[2023-10-15 16:05:10,795][52866] Updated weights for policy 1, policy_version 32320 (0.0010) -[2023-10-15 16:05:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 66093056. Throughput: 0: 1802.2, 1: 1793.1. Samples: 16538776. Policy #0 lag: (min: 14.0, avg: 28.3, max: 46.0) -[2023-10-15 16:05:13,442][51532] Avg episode reward: [(0, '35.920'), (1, '35.550')] -[2023-10-15 16:05:13,445][52833] Updated weights for policy 0, policy_version 32230 (0.0009) -[2023-10-15 16:05:13,813][52833] Updated weights for policy 0, policy_version 32240 (0.0009) -[2023-10-15 16:05:14,188][52833] Updated weights for policy 0, policy_version 32250 (0.0007) -[2023-10-15 16:05:14,506][52866] Updated weights for policy 1, policy_version 32330 (0.0007) -[2023-10-15 16:05:14,875][52866] Updated weights for policy 1, policy_version 32340 (0.0007) -[2023-10-15 16:05:15,236][52866] Updated weights for policy 1, policy_version 32350 (0.0007) -[2023-10-15 16:05:17,937][52833] Updated weights for policy 0, policy_version 32260 (0.0007) -[2023-10-15 16:05:18,314][52833] Updated weights for policy 0, policy_version 32270 (0.0008) -[2023-10-15 16:05:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 66158592. Throughput: 0: 1790.6, 1: 1793.9. Samples: 16548644. Policy #0 lag: (min: 14.0, avg: 28.3, max: 46.0) -[2023-10-15 16:05:18,442][51532] Avg episode reward: [(0, '38.620'), (1, '35.260')] -[2023-10-15 16:05:18,675][52833] Updated weights for policy 0, policy_version 32280 (0.0007) -[2023-10-15 16:05:18,983][52866] Updated weights for policy 1, policy_version 32360 (0.0008) -[2023-10-15 16:05:19,351][52866] Updated weights for policy 1, policy_version 32370 (0.0007) -[2023-10-15 16:05:19,713][52866] Updated weights for policy 1, policy_version 32380 (0.0008) -[2023-10-15 16:05:22,475][52833] Updated weights for policy 0, policy_version 32290 (0.0010) -[2023-10-15 16:05:22,846][52833] Updated weights for policy 0, policy_version 32300 (0.0007) -[2023-10-15 16:05:23,228][52833] Updated weights for policy 0, policy_version 32310 (0.0008) -[2023-10-15 16:05:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 66224128. Throughput: 0: 1801.3, 1: 1795.2. Samples: 16571184. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:05:23,441][51532] Avg episode reward: [(0, '38.790'), (1, '37.310')] -[2023-10-15 16:05:23,535][52866] Updated weights for policy 1, policy_version 32390 (0.0008) -[2023-10-15 16:05:23,589][52833] Updated weights for policy 0, policy_version 32320 (0.0008) -[2023-10-15 16:05:23,903][52866] Updated weights for policy 1, policy_version 32400 (0.0009) -[2023-10-15 16:05:24,261][52866] Updated weights for policy 1, policy_version 32410 (0.0009) -[2023-10-15 16:05:27,324][52833] Updated weights for policy 0, policy_version 32330 (0.0008) -[2023-10-15 16:05:27,694][52833] Updated weights for policy 0, policy_version 32340 (0.0008) -[2023-10-15 16:05:28,018][52866] Updated weights for policy 1, policy_version 32420 (0.0009) -[2023-10-15 16:05:28,060][52833] Updated weights for policy 0, policy_version 32350 (0.0007) -[2023-10-15 16:05:28,390][52866] Updated weights for policy 1, policy_version 32430 (0.0011) -[2023-10-15 16:05:28,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 66322432. Throughput: 0: 1802.5, 1: 1806.5. Samples: 16592372. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:05:28,441][51532] Avg episode reward: [(0, '40.120'), (1, '36.430')] -[2023-10-15 16:05:28,763][52866] Updated weights for policy 1, policy_version 32440 (0.0008) -[2023-10-15 16:05:31,894][52833] Updated weights for policy 0, policy_version 32360 (0.0008) -[2023-10-15 16:05:32,274][52833] Updated weights for policy 0, policy_version 32370 (0.0009) -[2023-10-15 16:05:32,475][52866] Updated weights for policy 1, policy_version 32450 (0.0007) -[2023-10-15 16:05:32,636][52833] Updated weights for policy 0, policy_version 32380 (0.0007) -[2023-10-15 16:05:32,843][52866] Updated weights for policy 1, policy_version 32460 (0.0007) -[2023-10-15 16:05:33,208][52866] Updated weights for policy 1, policy_version 32470 (0.0007) -[2023-10-15 16:05:33,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 66387968. Throughput: 0: 1795.0, 1: 1795.9. Samples: 16603380. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:05:33,441][51532] Avg episode reward: [(0, '40.290'), (1, '36.290')] -[2023-10-15 16:05:33,572][52866] Updated weights for policy 1, policy_version 32480 (0.0008) -[2023-10-15 16:05:36,346][52833] Updated weights for policy 0, policy_version 32390 (0.0008) -[2023-10-15 16:05:36,711][52833] Updated weights for policy 0, policy_version 32400 (0.0009) -[2023-10-15 16:05:37,074][52833] Updated weights for policy 0, policy_version 32410 (0.0008) -[2023-10-15 16:05:37,383][52866] Updated weights for policy 1, policy_version 32490 (0.0010) -[2023-10-15 16:05:37,753][52866] Updated weights for policy 1, policy_version 32500 (0.0009) -[2023-10-15 16:05:38,123][52866] Updated weights for policy 1, policy_version 32510 (0.0008) -[2023-10-15 16:05:38,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 66486272. Throughput: 0: 1800.6, 1: 1803.8. Samples: 16624744. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:05:38,442][51532] Avg episode reward: [(0, '40.240'), (1, '36.910')] -[2023-10-15 16:05:40,693][52833] Updated weights for policy 0, policy_version 32420 (0.0007) -[2023-10-15 16:05:41,058][52833] Updated weights for policy 0, policy_version 32430 (0.0008) -[2023-10-15 16:05:41,434][52833] Updated weights for policy 0, policy_version 32440 (0.0009) -[2023-10-15 16:05:41,936][52866] Updated weights for policy 1, policy_version 32520 (0.0009) -[2023-10-15 16:05:42,299][52866] Updated weights for policy 1, policy_version 32530 (0.0007) -[2023-10-15 16:05:42,673][52866] Updated weights for policy 1, policy_version 32540 (0.0008) -[2023-10-15 16:05:43,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 66551808. Throughput: 0: 1782.9, 1: 1794.1. Samples: 16644958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:05:43,442][51532] Avg episode reward: [(0, '41.170'), (1, '35.770')] -[2023-10-15 16:05:43,455][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000032448_33226752.pth... -[2023-10-15 16:05:43,455][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000032544_33325056.pth... -[2023-10-15 16:05:43,487][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000030752_31490048.pth -[2023-10-15 16:05:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000030848_31588352.pth -[2023-10-15 16:05:45,231][52833] Updated weights for policy 0, policy_version 32450 (0.0008) -[2023-10-15 16:05:45,658][52833] Updated weights for policy 0, policy_version 32460 (0.0010) -[2023-10-15 16:05:46,021][52833] Updated weights for policy 0, policy_version 32470 (0.0011) -[2023-10-15 16:05:46,333][52866] Updated weights for policy 1, policy_version 32550 (0.0008) -[2023-10-15 16:05:46,401][52833] Updated weights for policy 0, policy_version 32480 (0.0008) -[2023-10-15 16:05:46,698][52866] Updated weights for policy 1, policy_version 32560 (0.0010) -[2023-10-15 16:05:47,062][52866] Updated weights for policy 1, policy_version 32570 (0.0010) -[2023-10-15 16:05:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66617344. Throughput: 0: 1799.3, 1: 1801.5. Samples: 16656716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:05:48,441][51532] Avg episode reward: [(0, '40.340'), (1, '35.510')] -[2023-10-15 16:05:50,285][52833] Updated weights for policy 0, policy_version 32490 (0.0008) -[2023-10-15 16:05:50,652][52833] Updated weights for policy 0, policy_version 32500 (0.0008) -[2023-10-15 16:05:50,845][52866] Updated weights for policy 1, policy_version 32580 (0.0008) -[2023-10-15 16:05:51,027][52833] Updated weights for policy 0, policy_version 32510 (0.0009) -[2023-10-15 16:05:51,203][52866] Updated weights for policy 1, policy_version 32590 (0.0008) -[2023-10-15 16:05:51,579][52866] Updated weights for policy 1, policy_version 32600 (0.0009) -[2023-10-15 16:05:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66682880. Throughput: 0: 1778.9, 1: 1790.1. Samples: 16676716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:05:53,441][51532] Avg episode reward: [(0, '42.010'), (1, '36.560')] -[2023-10-15 16:05:54,782][52833] Updated weights for policy 0, policy_version 32520 (0.0009) -[2023-10-15 16:05:55,149][52833] Updated weights for policy 0, policy_version 32530 (0.0010) -[2023-10-15 16:05:55,307][52866] Updated weights for policy 1, policy_version 32610 (0.0008) -[2023-10-15 16:05:55,523][52833] Updated weights for policy 0, policy_version 32540 (0.0009) -[2023-10-15 16:05:55,670][52866] Updated weights for policy 1, policy_version 32620 (0.0008) -[2023-10-15 16:05:56,043][52866] Updated weights for policy 1, policy_version 32630 (0.0008) -[2023-10-15 16:05:56,409][52866] Updated weights for policy 1, policy_version 32640 (0.0009) -[2023-10-15 16:05:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 66748416. Throughput: 0: 1770.9, 1: 1796.6. Samples: 16699314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:05:58,442][51532] Avg episode reward: [(0, '44.400'), (1, '34.170')] -[2023-10-15 16:05:58,453][52410] Saving new best policy, reward=44.400! -[2023-10-15 16:05:59,480][52833] Updated weights for policy 0, policy_version 32550 (0.0008) -[2023-10-15 16:05:59,860][52833] Updated weights for policy 0, policy_version 32560 (0.0009) -[2023-10-15 16:06:00,192][52866] Updated weights for policy 1, policy_version 32650 (0.0008) -[2023-10-15 16:06:00,233][52833] Updated weights for policy 0, policy_version 32570 (0.0009) -[2023-10-15 16:06:00,557][52866] Updated weights for policy 1, policy_version 32660 (0.0007) -[2023-10-15 16:06:00,920][52866] Updated weights for policy 1, policy_version 32670 (0.0011) -[2023-10-15 16:06:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 66813952. Throughput: 0: 1770.8, 1: 1797.9. Samples: 16709232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:06:03,441][51532] Avg episode reward: [(0, '44.550'), (1, '34.630')] -[2023-10-15 16:06:03,442][52410] Saving new best policy, reward=44.550! -[2023-10-15 16:06:03,809][52833] Updated weights for policy 0, policy_version 32580 (0.0008) -[2023-10-15 16:06:04,170][52833] Updated weights for policy 0, policy_version 32590 (0.0011) -[2023-10-15 16:06:04,546][52833] Updated weights for policy 0, policy_version 32600 (0.0008) -[2023-10-15 16:06:04,619][52866] Updated weights for policy 1, policy_version 32680 (0.0007) -[2023-10-15 16:06:04,985][52866] Updated weights for policy 1, policy_version 32690 (0.0008) -[2023-10-15 16:06:05,360][52866] Updated weights for policy 1, policy_version 32700 (0.0009) -[2023-10-15 16:06:08,278][52833] Updated weights for policy 0, policy_version 32610 (0.0008) -[2023-10-15 16:06:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 66879488. Throughput: 0: 1767.5, 1: 1792.5. Samples: 16731382. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:06:08,441][51532] Avg episode reward: [(0, '41.330'), (1, '34.030')] -[2023-10-15 16:06:08,647][52833] Updated weights for policy 0, policy_version 32620 (0.0008) -[2023-10-15 16:06:09,022][52833] Updated weights for policy 0, policy_version 32630 (0.0008) -[2023-10-15 16:06:09,108][52866] Updated weights for policy 1, policy_version 32710 (0.0009) -[2023-10-15 16:06:09,391][52833] Updated weights for policy 0, policy_version 32640 (0.0009) -[2023-10-15 16:06:09,473][52866] Updated weights for policy 1, policy_version 32720 (0.0008) -[2023-10-15 16:06:09,845][52866] Updated weights for policy 1, policy_version 32730 (0.0008) -[2023-10-15 16:06:13,055][52833] Updated weights for policy 0, policy_version 32650 (0.0009) -[2023-10-15 16:06:13,422][52833] Updated weights for policy 0, policy_version 32660 (0.0008) -[2023-10-15 16:06:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 66945024. Throughput: 0: 1787.5, 1: 1796.8. Samples: 16753668. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:06:13,442][51532] Avg episode reward: [(0, '40.850'), (1, '35.220')] -[2023-10-15 16:06:13,639][52866] Updated weights for policy 1, policy_version 32740 (0.0009) -[2023-10-15 16:06:13,802][52833] Updated weights for policy 0, policy_version 32670 (0.0008) -[2023-10-15 16:06:14,005][52866] Updated weights for policy 1, policy_version 32750 (0.0007) -[2023-10-15 16:06:14,370][52866] Updated weights for policy 1, policy_version 32760 (0.0008) -[2023-10-15 16:06:17,687][52833] Updated weights for policy 0, policy_version 32680 (0.0007) -[2023-10-15 16:06:18,044][52866] Updated weights for policy 1, policy_version 32770 (0.0009) -[2023-10-15 16:06:18,050][52833] Updated weights for policy 0, policy_version 32690 (0.0007) -[2023-10-15 16:06:18,409][52866] Updated weights for policy 1, policy_version 32780 (0.0010) -[2023-10-15 16:06:18,423][52833] Updated weights for policy 0, policy_version 32700 (0.0007) -[2023-10-15 16:06:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 67010560. Throughput: 0: 1769.2, 1: 1793.1. Samples: 16763684. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:06:18,442][51532] Avg episode reward: [(0, '39.840'), (1, '33.490')] -[2023-10-15 16:06:18,786][52866] Updated weights for policy 1, policy_version 32790 (0.0009) -[2023-10-15 16:06:19,151][52866] Updated weights for policy 1, policy_version 32800 (0.0009) -[2023-10-15 16:06:22,184][52833] Updated weights for policy 0, policy_version 32710 (0.0008) -[2023-10-15 16:06:22,548][52833] Updated weights for policy 0, policy_version 32720 (0.0009) -[2023-10-15 16:06:22,902][52866] Updated weights for policy 1, policy_version 32810 (0.0009) -[2023-10-15 16:06:22,924][52833] Updated weights for policy 0, policy_version 32730 (0.0007) -[2023-10-15 16:06:23,276][52866] Updated weights for policy 1, policy_version 32820 (0.0008) -[2023-10-15 16:06:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 67108864. Throughput: 0: 1790.5, 1: 1792.7. Samples: 16785990. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:06:23,441][51532] Avg episode reward: [(0, '39.820'), (1, '33.290')] -[2023-10-15 16:06:23,635][52866] Updated weights for policy 1, policy_version 32830 (0.0008) -[2023-10-15 16:06:26,719][52833] Updated weights for policy 0, policy_version 32740 (0.0008) -[2023-10-15 16:06:27,097][52833] Updated weights for policy 0, policy_version 32750 (0.0008) -[2023-10-15 16:06:27,461][52833] Updated weights for policy 0, policy_version 32760 (0.0008) -[2023-10-15 16:06:27,507][52866] Updated weights for policy 1, policy_version 32840 (0.0008) -[2023-10-15 16:06:27,877][52866] Updated weights for policy 1, policy_version 32850 (0.0007) -[2023-10-15 16:06:28,242][52866] Updated weights for policy 1, policy_version 32860 (0.0007) -[2023-10-15 16:06:28,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 67207168. Throughput: 0: 1772.9, 1: 1806.3. Samples: 16806022. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) -[2023-10-15 16:06:28,442][51532] Avg episode reward: [(0, '41.020'), (1, '32.490')] -[2023-10-15 16:06:31,268][52833] Updated weights for policy 0, policy_version 32770 (0.0008) -[2023-10-15 16:06:31,688][52833] Updated weights for policy 0, policy_version 32780 (0.0008) -[2023-10-15 16:06:31,982][52866] Updated weights for policy 1, policy_version 32870 (0.0008) -[2023-10-15 16:06:32,053][52833] Updated weights for policy 0, policy_version 32790 (0.0008) -[2023-10-15 16:06:32,349][52866] Updated weights for policy 1, policy_version 32880 (0.0007) -[2023-10-15 16:06:32,415][52833] Updated weights for policy 0, policy_version 32800 (0.0009) -[2023-10-15 16:06:32,711][52866] Updated weights for policy 1, policy_version 32890 (0.0008) -[2023-10-15 16:06:33,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 67272704. Throughput: 0: 1796.5, 1: 1789.6. Samples: 16818094. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) -[2023-10-15 16:06:33,442][51532] Avg episode reward: [(0, '36.950'), (1, '31.950')] -[2023-10-15 16:06:36,103][52833] Updated weights for policy 0, policy_version 32810 (0.0009) -[2023-10-15 16:06:36,478][52833] Updated weights for policy 0, policy_version 32820 (0.0008) -[2023-10-15 16:06:36,568][52866] Updated weights for policy 1, policy_version 32900 (0.0010) -[2023-10-15 16:06:36,842][52833] Updated weights for policy 0, policy_version 32830 (0.0007) -[2023-10-15 16:06:36,932][52866] Updated weights for policy 1, policy_version 32910 (0.0007) -[2023-10-15 16:06:37,296][52866] Updated weights for policy 1, policy_version 32920 (0.0009) -[2023-10-15 16:06:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67338240. Throughput: 0: 1781.2, 1: 1810.1. Samples: 16838326. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) -[2023-10-15 16:06:38,442][51532] Avg episode reward: [(0, '35.070'), (1, '33.260')] -[2023-10-15 16:06:40,521][52833] Updated weights for policy 0, policy_version 32840 (0.0010) -[2023-10-15 16:06:40,885][52833] Updated weights for policy 0, policy_version 32850 (0.0008) -[2023-10-15 16:06:41,011][52866] Updated weights for policy 1, policy_version 32930 (0.0009) -[2023-10-15 16:06:41,262][52833] Updated weights for policy 0, policy_version 32860 (0.0008) -[2023-10-15 16:06:41,378][52866] Updated weights for policy 1, policy_version 32940 (0.0009) -[2023-10-15 16:06:41,737][52866] Updated weights for policy 1, policy_version 32950 (0.0010) -[2023-10-15 16:06:42,106][52866] Updated weights for policy 1, policy_version 32960 (0.0010) -[2023-10-15 16:06:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 67403776. Throughput: 0: 1783.7, 1: 1785.1. Samples: 16859912. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) -[2023-10-15 16:06:43,442][51532] Avg episode reward: [(0, '39.360'), (1, '33.470')] -[2023-10-15 16:06:45,098][52833] Updated weights for policy 0, policy_version 32870 (0.0008) -[2023-10-15 16:06:45,469][52833] Updated weights for policy 0, policy_version 32880 (0.0007) -[2023-10-15 16:06:45,838][52833] Updated weights for policy 0, policy_version 32890 (0.0008) -[2023-10-15 16:06:46,046][52866] Updated weights for policy 1, policy_version 32970 (0.0007) -[2023-10-15 16:06:46,413][52866] Updated weights for policy 1, policy_version 32980 (0.0009) -[2023-10-15 16:06:46,788][52866] Updated weights for policy 1, policy_version 32990 (0.0008) -[2023-10-15 16:06:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67469312. Throughput: 0: 1788.4, 1: 1803.6. Samples: 16870874. Policy #0 lag: (min: 12.0, avg: 15.0, max: 44.0) -[2023-10-15 16:06:48,441][51532] Avg episode reward: [(0, '37.590'), (1, '35.080')] -[2023-10-15 16:06:49,615][52833] Updated weights for policy 0, policy_version 32900 (0.0009) -[2023-10-15 16:06:49,983][52833] Updated weights for policy 0, policy_version 32910 (0.0009) -[2023-10-15 16:06:50,356][52833] Updated weights for policy 0, policy_version 32920 (0.0008) -[2023-10-15 16:06:50,581][52866] Updated weights for policy 1, policy_version 33000 (0.0008) -[2023-10-15 16:06:50,948][52866] Updated weights for policy 1, policy_version 33010 (0.0010) -[2023-10-15 16:06:51,316][52866] Updated weights for policy 1, policy_version 33020 (0.0010) -[2023-10-15 16:06:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 67534848. Throughput: 0: 1787.2, 1: 1779.2. Samples: 16891872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:06:53,442][51532] Avg episode reward: [(0, '37.700'), (1, '34.840')] -[2023-10-15 16:06:54,043][52833] Updated weights for policy 0, policy_version 32930 (0.0007) -[2023-10-15 16:06:54,412][52833] Updated weights for policy 0, policy_version 32940 (0.0009) -[2023-10-15 16:06:54,784][52833] Updated weights for policy 0, policy_version 32950 (0.0008) -[2023-10-15 16:06:54,875][52866] Updated weights for policy 1, policy_version 33030 (0.0009) -[2023-10-15 16:06:55,143][52833] Updated weights for policy 0, policy_version 32960 (0.0008) -[2023-10-15 16:06:55,239][52866] Updated weights for policy 1, policy_version 33040 (0.0008) -[2023-10-15 16:06:55,602][52866] Updated weights for policy 1, policy_version 33050 (0.0009) -[2023-10-15 16:06:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67600384. Throughput: 0: 1792.7, 1: 1780.6. Samples: 16914464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:06:58,441][51532] Avg episode reward: [(0, '40.020'), (1, '34.570')] -[2023-10-15 16:06:58,891][52833] Updated weights for policy 0, policy_version 32970 (0.0007) -[2023-10-15 16:06:59,254][52833] Updated weights for policy 0, policy_version 32980 (0.0007) -[2023-10-15 16:06:59,374][52866] Updated weights for policy 1, policy_version 33060 (0.0007) -[2023-10-15 16:06:59,614][52833] Updated weights for policy 0, policy_version 32990 (0.0007) -[2023-10-15 16:06:59,738][52866] Updated weights for policy 1, policy_version 33070 (0.0008) -[2023-10-15 16:07:00,107][52866] Updated weights for policy 1, policy_version 33080 (0.0009) -[2023-10-15 16:07:03,326][52833] Updated weights for policy 0, policy_version 33000 (0.0007) -[2023-10-15 16:07:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 67665920. Throughput: 0: 1788.3, 1: 1781.4. Samples: 16924318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:07:03,442][51532] Avg episode reward: [(0, '37.670'), (1, '36.300')] -[2023-10-15 16:07:03,689][52833] Updated weights for policy 0, policy_version 33010 (0.0009) -[2023-10-15 16:07:03,866][52866] Updated weights for policy 1, policy_version 33090 (0.0009) -[2023-10-15 16:07:04,051][52833] Updated weights for policy 0, policy_version 33020 (0.0010) -[2023-10-15 16:07:04,222][52866] Updated weights for policy 1, policy_version 33100 (0.0009) -[2023-10-15 16:07:04,582][52866] Updated weights for policy 1, policy_version 33110 (0.0009) -[2023-10-15 16:07:04,954][52866] Updated weights for policy 1, policy_version 33120 (0.0011) -[2023-10-15 16:07:07,783][52833] Updated weights for policy 0, policy_version 33030 (0.0009) -[2023-10-15 16:07:08,152][52833] Updated weights for policy 0, policy_version 33040 (0.0009) -[2023-10-15 16:07:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 67731456. Throughput: 0: 1793.0, 1: 1781.5. Samples: 16946840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:07:08,442][51532] Avg episode reward: [(0, '36.040'), (1, '36.150')] -[2023-10-15 16:07:08,536][52833] Updated weights for policy 0, policy_version 33050 (0.0008) -[2023-10-15 16:07:08,974][52866] Updated weights for policy 1, policy_version 33130 (0.0007) -[2023-10-15 16:07:09,346][52866] Updated weights for policy 1, policy_version 33140 (0.0011) -[2023-10-15 16:07:09,714][52866] Updated weights for policy 1, policy_version 33150 (0.0009) -[2023-10-15 16:07:12,347][52833] Updated weights for policy 0, policy_version 33060 (0.0010) -[2023-10-15 16:07:12,715][52833] Updated weights for policy 0, policy_version 33070 (0.0011) -[2023-10-15 16:07:13,088][52833] Updated weights for policy 0, policy_version 33080 (0.0010) -[2023-10-15 16:07:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 67829760. Throughput: 0: 1801.5, 1: 1801.5. Samples: 16968156. Policy #0 lag: (min: 19.0, avg: 44.6, max: 48.0) -[2023-10-15 16:07:13,442][51532] Avg episode reward: [(0, '38.020'), (1, '37.770')] -[2023-10-15 16:07:13,570][52866] Updated weights for policy 1, policy_version 33160 (0.0009) -[2023-10-15 16:07:13,945][52866] Updated weights for policy 1, policy_version 33170 (0.0011) -[2023-10-15 16:07:14,311][52866] Updated weights for policy 1, policy_version 33180 (0.0010) -[2023-10-15 16:07:16,930][52833] Updated weights for policy 0, policy_version 33090 (0.0007) -[2023-10-15 16:07:17,328][52833] Updated weights for policy 0, policy_version 33100 (0.0007) -[2023-10-15 16:07:17,690][52833] Updated weights for policy 0, policy_version 33110 (0.0007) -[2023-10-15 16:07:18,046][52866] Updated weights for policy 1, policy_version 33190 (0.0008) -[2023-10-15 16:07:18,060][52833] Updated weights for policy 0, policy_version 33120 (0.0007) -[2023-10-15 16:07:18,419][52866] Updated weights for policy 1, policy_version 33200 (0.0009) -[2023-10-15 16:07:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 67895296. Throughput: 0: 1787.0, 1: 1779.9. Samples: 16978602. Policy #0 lag: (min: 19.0, avg: 44.6, max: 48.0) -[2023-10-15 16:07:18,442][51532] Avg episode reward: [(0, '37.130'), (1, '38.940')] -[2023-10-15 16:07:18,789][52866] Updated weights for policy 1, policy_version 33210 (0.0007) -[2023-10-15 16:07:21,827][52833] Updated weights for policy 0, policy_version 33130 (0.0009) -[2023-10-15 16:07:22,197][52833] Updated weights for policy 0, policy_version 33140 (0.0010) -[2023-10-15 16:07:22,494][52866] Updated weights for policy 1, policy_version 33220 (0.0008) -[2023-10-15 16:07:22,565][52833] Updated weights for policy 0, policy_version 33150 (0.0007) -[2023-10-15 16:07:22,852][52866] Updated weights for policy 1, policy_version 33230 (0.0008) -[2023-10-15 16:07:23,227][52866] Updated weights for policy 1, policy_version 33240 (0.0008) -[2023-10-15 16:07:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 67960832. Throughput: 0: 1806.9, 1: 1799.6. Samples: 17000620. Policy #0 lag: (min: 19.0, avg: 44.6, max: 48.0) -[2023-10-15 16:07:23,442][51532] Avg episode reward: [(0, '35.470'), (1, '40.310')] -[2023-10-15 16:07:26,201][52833] Updated weights for policy 0, policy_version 33160 (0.0008) -[2023-10-15 16:07:26,579][52833] Updated weights for policy 0, policy_version 33170 (0.0008) -[2023-10-15 16:07:26,948][52833] Updated weights for policy 0, policy_version 33180 (0.0008) -[2023-10-15 16:07:26,965][52866] Updated weights for policy 1, policy_version 33250 (0.0011) -[2023-10-15 16:07:27,312][52866] Updated weights for policy 1, policy_version 33260 (0.0010) -[2023-10-15 16:07:27,683][52866] Updated weights for policy 1, policy_version 33270 (0.0008) -[2023-10-15 16:07:28,039][52866] Updated weights for policy 1, policy_version 33280 (0.0009) -[2023-10-15 16:07:28,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68059136. Throughput: 0: 1793.0, 1: 1785.7. Samples: 17020952. Policy #0 lag: (min: 19.0, avg: 44.6, max: 48.0) -[2023-10-15 16:07:28,442][51532] Avg episode reward: [(0, '36.510'), (1, '41.150')] -[2023-10-15 16:07:30,689][52833] Updated weights for policy 0, policy_version 33190 (0.0007) -[2023-10-15 16:07:31,067][52833] Updated weights for policy 0, policy_version 33200 (0.0011) -[2023-10-15 16:07:31,426][52833] Updated weights for policy 0, policy_version 33210 (0.0010) -[2023-10-15 16:07:31,755][52866] Updated weights for policy 1, policy_version 33290 (0.0010) -[2023-10-15 16:07:32,131][52866] Updated weights for policy 1, policy_version 33300 (0.0009) -[2023-10-15 16:07:32,501][52866] Updated weights for policy 1, policy_version 33310 (0.0008) -[2023-10-15 16:07:33,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 68124672. Throughput: 0: 1812.7, 1: 1795.4. Samples: 17033238. Policy #0 lag: (min: 19.0, avg: 44.6, max: 48.0) -[2023-10-15 16:07:33,441][51532] Avg episode reward: [(0, '33.980'), (1, '41.740')] -[2023-10-15 16:07:35,203][52833] Updated weights for policy 0, policy_version 33220 (0.0008) -[2023-10-15 16:07:35,574][52833] Updated weights for policy 0, policy_version 33230 (0.0010) -[2023-10-15 16:07:35,938][52833] Updated weights for policy 0, policy_version 33240 (0.0010) -[2023-10-15 16:07:36,274][52866] Updated weights for policy 1, policy_version 33320 (0.0009) -[2023-10-15 16:07:36,638][52866] Updated weights for policy 1, policy_version 33330 (0.0010) -[2023-10-15 16:07:37,000][52866] Updated weights for policy 1, policy_version 33340 (0.0009) -[2023-10-15 16:07:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 68190208. Throughput: 0: 1798.9, 1: 1796.1. Samples: 17053644. Policy #0 lag: (min: 20.0, avg: 20.4, max: 33.0) -[2023-10-15 16:07:38,441][51532] Avg episode reward: [(0, '35.280'), (1, '39.390')] -[2023-10-15 16:07:39,628][52833] Updated weights for policy 0, policy_version 33250 (0.0009) -[2023-10-15 16:07:39,997][52833] Updated weights for policy 0, policy_version 33260 (0.0009) -[2023-10-15 16:07:40,365][52833] Updated weights for policy 0, policy_version 33270 (0.0010) -[2023-10-15 16:07:40,734][52833] Updated weights for policy 0, policy_version 33280 (0.0009) -[2023-10-15 16:07:40,737][52866] Updated weights for policy 1, policy_version 33350 (0.0008) -[2023-10-15 16:07:41,110][52866] Updated weights for policy 1, policy_version 33360 (0.0007) -[2023-10-15 16:07:41,474][52866] Updated weights for policy 1, policy_version 33370 (0.0009) -[2023-10-15 16:07:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 68255744. Throughput: 0: 1800.5, 1: 1783.9. Samples: 17075762. Policy #0 lag: (min: 20.0, avg: 20.4, max: 33.0) -[2023-10-15 16:07:43,442][51532] Avg episode reward: [(0, '35.580'), (1, '40.210')] -[2023-10-15 16:07:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000033376_34177024.pth... -[2023-10-15 16:07:43,455][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000033280_34078720.pth... -[2023-10-15 16:07:43,490][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000031616_32374784.pth -[2023-10-15 16:07:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000031712_32473088.pth -[2023-10-15 16:07:43,494][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000033280_34078720.pth -[2023-10-15 16:07:43,495][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000033376_34177024.pth -[2023-10-15 16:07:44,433][52833] Updated weights for policy 0, policy_version 33290 (0.0010) -[2023-10-15 16:07:44,801][52833] Updated weights for policy 0, policy_version 33300 (0.0010) -[2023-10-15 16:07:45,172][52833] Updated weights for policy 0, policy_version 33310 (0.0008) -[2023-10-15 16:07:45,303][52866] Updated weights for policy 1, policy_version 33380 (0.0007) -[2023-10-15 16:07:45,675][52866] Updated weights for policy 1, policy_version 33390 (0.0007) -[2023-10-15 16:07:46,031][52866] Updated weights for policy 1, policy_version 33400 (0.0007) -[2023-10-15 16:07:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 68321280. Throughput: 0: 1798.7, 1: 1791.3. Samples: 17085866. Policy #0 lag: (min: 20.0, avg: 20.4, max: 33.0) -[2023-10-15 16:07:48,442][51532] Avg episode reward: [(0, '37.490'), (1, '40.440')] -[2023-10-15 16:07:48,956][52833] Updated weights for policy 0, policy_version 33320 (0.0008) -[2023-10-15 16:07:49,320][52833] Updated weights for policy 0, policy_version 33330 (0.0007) -[2023-10-15 16:07:49,695][52833] Updated weights for policy 0, policy_version 33340 (0.0007) -[2023-10-15 16:07:49,755][52866] Updated weights for policy 1, policy_version 33410 (0.0008) -[2023-10-15 16:07:50,126][52866] Updated weights for policy 1, policy_version 33420 (0.0008) -[2023-10-15 16:07:50,488][52866] Updated weights for policy 1, policy_version 33430 (0.0007) -[2023-10-15 16:07:50,857][52866] Updated weights for policy 1, policy_version 33440 (0.0010) -[2023-10-15 16:07:53,256][52833] Updated weights for policy 0, policy_version 33350 (0.0010) -[2023-10-15 16:07:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 68386816. Throughput: 0: 1790.6, 1: 1782.9. Samples: 17107648. Policy #0 lag: (min: 20.0, avg: 20.4, max: 33.0) -[2023-10-15 16:07:53,441][51532] Avg episode reward: [(0, '37.650'), (1, '40.970')] -[2023-10-15 16:07:53,628][52833] Updated weights for policy 0, policy_version 33360 (0.0007) -[2023-10-15 16:07:54,000][52833] Updated weights for policy 0, policy_version 33370 (0.0008) -[2023-10-15 16:07:54,570][52866] Updated weights for policy 1, policy_version 33450 (0.0007) -[2023-10-15 16:07:54,947][52866] Updated weights for policy 1, policy_version 33460 (0.0009) -[2023-10-15 16:07:55,309][52866] Updated weights for policy 1, policy_version 33470 (0.0009) -[2023-10-15 16:07:57,656][52833] Updated weights for policy 0, policy_version 33380 (0.0009) -[2023-10-15 16:07:58,033][52833] Updated weights for policy 0, policy_version 33390 (0.0010) -[2023-10-15 16:07:58,401][52833] Updated weights for policy 0, policy_version 33400 (0.0007) -[2023-10-15 16:07:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 68452352. Throughput: 0: 1806.5, 1: 1786.4. Samples: 17129834. Policy #0 lag: (min: 20.0, avg: 20.4, max: 33.0) -[2023-10-15 16:07:58,441][51532] Avg episode reward: [(0, '36.370'), (1, '40.710')] -[2023-10-15 16:07:59,074][52866] Updated weights for policy 1, policy_version 33480 (0.0009) -[2023-10-15 16:07:59,442][52866] Updated weights for policy 1, policy_version 33490 (0.0008) -[2023-10-15 16:07:59,811][52866] Updated weights for policy 1, policy_version 33500 (0.0008) -[2023-10-15 16:08:02,241][52833] Updated weights for policy 0, policy_version 33410 (0.0009) -[2023-10-15 16:08:02,648][52833] Updated weights for policy 0, policy_version 33420 (0.0009) -[2023-10-15 16:08:03,011][52833] Updated weights for policy 0, policy_version 33430 (0.0009) -[2023-10-15 16:08:03,372][52833] Updated weights for policy 0, policy_version 33440 (0.0009) -[2023-10-15 16:08:03,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 68550656. Throughput: 0: 1799.8, 1: 1787.9. Samples: 17140048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:08:03,442][51532] Avg episode reward: [(0, '36.490'), (1, '38.970')] -[2023-10-15 16:08:03,447][52866] Updated weights for policy 1, policy_version 33510 (0.0008) -[2023-10-15 16:08:03,812][52866] Updated weights for policy 1, policy_version 33520 (0.0008) -[2023-10-15 16:08:04,175][52866] Updated weights for policy 1, policy_version 33530 (0.0007) -[2023-10-15 16:08:07,037][52833] Updated weights for policy 0, policy_version 33450 (0.0008) -[2023-10-15 16:08:07,407][52833] Updated weights for policy 0, policy_version 33460 (0.0009) -[2023-10-15 16:08:07,786][52833] Updated weights for policy 0, policy_version 33470 (0.0007) -[2023-10-15 16:08:07,834][52866] Updated weights for policy 1, policy_version 33540 (0.0007) -[2023-10-15 16:08:08,196][52866] Updated weights for policy 1, policy_version 33550 (0.0007) -[2023-10-15 16:08:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 68616192. Throughput: 0: 1804.9, 1: 1791.3. Samples: 17162448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:08:08,442][51532] Avg episode reward: [(0, '35.760'), (1, '39.540')] -[2023-10-15 16:08:08,555][52866] Updated weights for policy 1, policy_version 33560 (0.0007) -[2023-10-15 16:08:11,616][52833] Updated weights for policy 0, policy_version 33480 (0.0008) -[2023-10-15 16:08:11,992][52833] Updated weights for policy 0, policy_version 33490 (0.0008) -[2023-10-15 16:08:12,366][52833] Updated weights for policy 0, policy_version 33500 (0.0007) -[2023-10-15 16:08:12,480][52866] Updated weights for policy 1, policy_version 33570 (0.0008) -[2023-10-15 16:08:12,846][52866] Updated weights for policy 1, policy_version 33580 (0.0007) -[2023-10-15 16:08:13,222][52866] Updated weights for policy 1, policy_version 33590 (0.0009) -[2023-10-15 16:08:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 68681728. Throughput: 0: 1792.5, 1: 1811.2. Samples: 17183116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:08:13,442][51532] Avg episode reward: [(0, '35.090'), (1, '39.520')] -[2023-10-15 16:08:13,588][52866] Updated weights for policy 1, policy_version 33600 (0.0007) -[2023-10-15 16:08:16,281][52833] Updated weights for policy 0, policy_version 33510 (0.0009) -[2023-10-15 16:08:16,659][52833] Updated weights for policy 0, policy_version 33520 (0.0008) -[2023-10-15 16:08:17,024][52833] Updated weights for policy 0, policy_version 33530 (0.0008) -[2023-10-15 16:08:17,373][52866] Updated weights for policy 1, policy_version 33610 (0.0007) -[2023-10-15 16:08:17,745][52866] Updated weights for policy 1, policy_version 33620 (0.0008) -[2023-10-15 16:08:18,118][52866] Updated weights for policy 1, policy_version 33630 (0.0010) -[2023-10-15 16:08:18,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 68780032. Throughput: 0: 1797.4, 1: 1793.7. Samples: 17194840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:08:18,442][51532] Avg episode reward: [(0, '34.980'), (1, '38.430')] -[2023-10-15 16:08:20,820][52833] Updated weights for policy 0, policy_version 33540 (0.0008) -[2023-10-15 16:08:21,187][52833] Updated weights for policy 0, policy_version 33550 (0.0010) -[2023-10-15 16:08:21,551][52833] Updated weights for policy 0, policy_version 33560 (0.0009) -[2023-10-15 16:08:21,894][52866] Updated weights for policy 1, policy_version 33640 (0.0008) -[2023-10-15 16:08:22,260][52866] Updated weights for policy 1, policy_version 33650 (0.0010) -[2023-10-15 16:08:22,625][52866] Updated weights for policy 1, policy_version 33660 (0.0007) -[2023-10-15 16:08:23,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 68845568. Throughput: 0: 1781.8, 1: 1814.0. Samples: 17215458. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:08:23,441][51532] Avg episode reward: [(0, '32.810'), (1, '40.950')] -[2023-10-15 16:08:25,260][52833] Updated weights for policy 0, policy_version 33570 (0.0009) -[2023-10-15 16:08:25,636][52833] Updated weights for policy 0, policy_version 33580 (0.0009) -[2023-10-15 16:08:26,007][52833] Updated weights for policy 0, policy_version 33590 (0.0007) -[2023-10-15 16:08:26,192][52866] Updated weights for policy 1, policy_version 33670 (0.0008) -[2023-10-15 16:08:26,369][52833] Updated weights for policy 0, policy_version 33600 (0.0010) -[2023-10-15 16:08:26,554][52866] Updated weights for policy 1, policy_version 33680 (0.0009) -[2023-10-15 16:08:26,928][52866] Updated weights for policy 1, policy_version 33690 (0.0008) -[2023-10-15 16:08:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68911104. Throughput: 0: 1781.7, 1: 1803.8. Samples: 17237108. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:08:28,442][51532] Avg episode reward: [(0, '33.700'), (1, '42.000')] -[2023-10-15 16:08:30,032][52833] Updated weights for policy 0, policy_version 33610 (0.0011) -[2023-10-15 16:08:30,410][52833] Updated weights for policy 0, policy_version 33620 (0.0010) -[2023-10-15 16:08:30,664][52866] Updated weights for policy 1, policy_version 33700 (0.0007) -[2023-10-15 16:08:30,773][52833] Updated weights for policy 0, policy_version 33630 (0.0009) -[2023-10-15 16:08:31,031][52866] Updated weights for policy 1, policy_version 33710 (0.0008) -[2023-10-15 16:08:31,393][52866] Updated weights for policy 1, policy_version 33720 (0.0009) -[2023-10-15 16:08:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68976640. Throughput: 0: 1783.0, 1: 1816.4. Samples: 17247836. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:08:33,441][51532] Avg episode reward: [(0, '33.860'), (1, '43.630')] -[2023-10-15 16:08:34,509][52833] Updated weights for policy 0, policy_version 33640 (0.0009) -[2023-10-15 16:08:34,880][52833] Updated weights for policy 0, policy_version 33650 (0.0009) -[2023-10-15 16:08:34,971][52866] Updated weights for policy 1, policy_version 33730 (0.0009) -[2023-10-15 16:08:35,245][52833] Updated weights for policy 0, policy_version 33660 (0.0007) -[2023-10-15 16:08:35,341][52866] Updated weights for policy 1, policy_version 33740 (0.0007) -[2023-10-15 16:08:35,703][52866] Updated weights for policy 1, policy_version 33750 (0.0007) -[2023-10-15 16:08:36,074][52866] Updated weights for policy 1, policy_version 33760 (0.0007) -[2023-10-15 16:08:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 69042176. Throughput: 0: 1789.5, 1: 1807.0. Samples: 17269490. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:08:38,442][51532] Avg episode reward: [(0, '35.280'), (1, '46.700')] -[2023-10-15 16:08:38,444][52518] Saving new best policy, reward=46.700! -[2023-10-15 16:08:39,091][52833] Updated weights for policy 0, policy_version 33670 (0.0007) -[2023-10-15 16:08:39,461][52833] Updated weights for policy 0, policy_version 33680 (0.0007) -[2023-10-15 16:08:39,826][52833] Updated weights for policy 0, policy_version 33690 (0.0008) -[2023-10-15 16:08:39,909][52866] Updated weights for policy 1, policy_version 33770 (0.0008) -[2023-10-15 16:08:40,266][52866] Updated weights for policy 1, policy_version 33780 (0.0007) -[2023-10-15 16:08:40,647][52866] Updated weights for policy 1, policy_version 33790 (0.0010) -[2023-10-15 16:08:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 69107712. Throughput: 0: 1797.9, 1: 1807.3. Samples: 17292068. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:08:43,442][51532] Avg episode reward: [(0, '34.950'), (1, '43.360')] -[2023-10-15 16:08:43,506][52833] Updated weights for policy 0, policy_version 33700 (0.0009) -[2023-10-15 16:08:43,873][52833] Updated weights for policy 0, policy_version 33710 (0.0009) -[2023-10-15 16:08:44,242][52833] Updated weights for policy 0, policy_version 33720 (0.0009) -[2023-10-15 16:08:44,502][52866] Updated weights for policy 1, policy_version 33800 (0.0008) -[2023-10-15 16:08:44,869][52866] Updated weights for policy 1, policy_version 33810 (0.0008) -[2023-10-15 16:08:45,232][52866] Updated weights for policy 1, policy_version 33820 (0.0008) -[2023-10-15 16:08:48,194][52833] Updated weights for policy 0, policy_version 33730 (0.0007) -[2023-10-15 16:08:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 69173248. Throughput: 0: 1782.5, 1: 1804.6. Samples: 17301468. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:08:48,441][51532] Avg episode reward: [(0, '35.560'), (1, '44.230')] -[2023-10-15 16:08:48,580][52833] Updated weights for policy 0, policy_version 33740 (0.0008) -[2023-10-15 16:08:48,911][52866] Updated weights for policy 1, policy_version 33830 (0.0009) -[2023-10-15 16:08:48,954][52833] Updated weights for policy 0, policy_version 33750 (0.0007) -[2023-10-15 16:08:49,278][52866] Updated weights for policy 1, policy_version 33840 (0.0008) -[2023-10-15 16:08:49,321][52833] Updated weights for policy 0, policy_version 33760 (0.0007) -[2023-10-15 16:08:49,645][52866] Updated weights for policy 1, policy_version 33850 (0.0008) -[2023-10-15 16:08:53,034][52833] Updated weights for policy 0, policy_version 33770 (0.0008) -[2023-10-15 16:08:53,402][52833] Updated weights for policy 0, policy_version 33780 (0.0010) -[2023-10-15 16:08:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 69238784. Throughput: 0: 1785.2, 1: 1790.0. Samples: 17323332. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 16:08:53,442][51532] Avg episode reward: [(0, '36.830'), (1, '44.570')] -[2023-10-15 16:08:53,651][52866] Updated weights for policy 1, policy_version 33860 (0.0009) -[2023-10-15 16:08:53,769][52833] Updated weights for policy 0, policy_version 33790 (0.0009) -[2023-10-15 16:08:54,019][52866] Updated weights for policy 1, policy_version 33870 (0.0009) -[2023-10-15 16:08:54,392][52866] Updated weights for policy 1, policy_version 33880 (0.0008) -[2023-10-15 16:08:57,535][52833] Updated weights for policy 0, policy_version 33800 (0.0008) -[2023-10-15 16:08:57,905][52833] Updated weights for policy 0, policy_version 33810 (0.0008) -[2023-10-15 16:08:57,936][52866] Updated weights for policy 1, policy_version 33890 (0.0009) -[2023-10-15 16:08:58,261][52833] Updated weights for policy 0, policy_version 33820 (0.0008) -[2023-10-15 16:08:58,303][52866] Updated weights for policy 1, policy_version 33900 (0.0009) -[2023-10-15 16:08:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 69337088. Throughput: 0: 1793.9, 1: 1805.0. Samples: 17345068. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 16:08:58,442][51532] Avg episode reward: [(0, '36.470'), (1, '42.680')] -[2023-10-15 16:08:58,675][52866] Updated weights for policy 1, policy_version 33910 (0.0008) -[2023-10-15 16:08:59,046][52866] Updated weights for policy 1, policy_version 33920 (0.0008) -[2023-10-15 16:09:01,973][52833] Updated weights for policy 0, policy_version 33830 (0.0008) -[2023-10-15 16:09:02,342][52833] Updated weights for policy 0, policy_version 33840 (0.0007) -[2023-10-15 16:09:02,707][52833] Updated weights for policy 0, policy_version 33850 (0.0007) -[2023-10-15 16:09:02,874][52866] Updated weights for policy 1, policy_version 33930 (0.0007) -[2023-10-15 16:09:03,238][52866] Updated weights for policy 1, policy_version 33940 (0.0008) -[2023-10-15 16:09:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 69402624. Throughput: 0: 1783.3, 1: 1794.8. Samples: 17355856. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 16:09:03,442][51532] Avg episode reward: [(0, '36.250'), (1, '44.250')] -[2023-10-15 16:09:03,612][52866] Updated weights for policy 1, policy_version 33950 (0.0008) -[2023-10-15 16:09:06,619][52833] Updated weights for policy 0, policy_version 33860 (0.0009) -[2023-10-15 16:09:06,988][52833] Updated weights for policy 0, policy_version 33870 (0.0007) -[2023-10-15 16:09:07,357][52833] Updated weights for policy 0, policy_version 33880 (0.0007) -[2023-10-15 16:09:07,428][52866] Updated weights for policy 1, policy_version 33960 (0.0009) -[2023-10-15 16:09:07,791][52866] Updated weights for policy 1, policy_version 33970 (0.0008) -[2023-10-15 16:09:08,162][52866] Updated weights for policy 1, policy_version 33980 (0.0008) -[2023-10-15 16:09:08,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 69500928. Throughput: 0: 1809.5, 1: 1796.2. Samples: 17377716. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 16:09:08,442][51532] Avg episode reward: [(0, '39.070'), (1, '42.500')] -[2023-10-15 16:09:11,118][52833] Updated weights for policy 0, policy_version 33890 (0.0009) -[2023-10-15 16:09:11,483][52833] Updated weights for policy 0, policy_version 33900 (0.0009) -[2023-10-15 16:09:11,775][52866] Updated weights for policy 1, policy_version 33990 (0.0007) -[2023-10-15 16:09:11,856][52833] Updated weights for policy 0, policy_version 33910 (0.0008) -[2023-10-15 16:09:12,139][52866] Updated weights for policy 1, policy_version 34000 (0.0007) -[2023-10-15 16:09:12,223][52833] Updated weights for policy 0, policy_version 33920 (0.0009) -[2023-10-15 16:09:12,501][52866] Updated weights for policy 1, policy_version 34010 (0.0010) -[2023-10-15 16:09:13,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 69566464. Throughput: 0: 1786.5, 1: 1787.6. Samples: 17397938. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:09:13,441][51532] Avg episode reward: [(0, '38.210'), (1, '42.350')] -[2023-10-15 16:09:15,836][52833] Updated weights for policy 0, policy_version 33930 (0.0009) -[2023-10-15 16:09:16,205][52833] Updated weights for policy 0, policy_version 33940 (0.0008) -[2023-10-15 16:09:16,255][52866] Updated weights for policy 1, policy_version 34020 (0.0008) -[2023-10-15 16:09:16,567][52833] Updated weights for policy 0, policy_version 33950 (0.0008) -[2023-10-15 16:09:16,623][52866] Updated weights for policy 1, policy_version 34030 (0.0007) -[2023-10-15 16:09:16,991][52866] Updated weights for policy 1, policy_version 34040 (0.0009) -[2023-10-15 16:09:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 69632000. Throughput: 0: 1808.4, 1: 1796.3. Samples: 17410048. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:09:18,441][51532] Avg episode reward: [(0, '37.810'), (1, '40.230')] -[2023-10-15 16:09:20,366][52833] Updated weights for policy 0, policy_version 33960 (0.0008) -[2023-10-15 16:09:20,734][52833] Updated weights for policy 0, policy_version 33970 (0.0008) -[2023-10-15 16:09:20,849][52866] Updated weights for policy 1, policy_version 34050 (0.0008) -[2023-10-15 16:09:21,108][52833] Updated weights for policy 0, policy_version 33980 (0.0010) -[2023-10-15 16:09:21,211][52866] Updated weights for policy 1, policy_version 34060 (0.0009) -[2023-10-15 16:09:21,572][52866] Updated weights for policy 1, policy_version 34070 (0.0007) -[2023-10-15 16:09:21,944][52866] Updated weights for policy 1, policy_version 34080 (0.0008) -[2023-10-15 16:09:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 69697536. Throughput: 0: 1784.8, 1: 1783.3. Samples: 17430056. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:09:23,442][51532] Avg episode reward: [(0, '38.890'), (1, '37.810')] -[2023-10-15 16:09:24,923][52833] Updated weights for policy 0, policy_version 33990 (0.0008) -[2023-10-15 16:09:25,282][52833] Updated weights for policy 0, policy_version 34000 (0.0010) -[2023-10-15 16:09:25,647][52833] Updated weights for policy 0, policy_version 34010 (0.0008) -[2023-10-15 16:09:25,744][52866] Updated weights for policy 1, policy_version 34090 (0.0008) -[2023-10-15 16:09:26,107][52866] Updated weights for policy 1, policy_version 34100 (0.0009) -[2023-10-15 16:09:26,470][52866] Updated weights for policy 1, policy_version 34110 (0.0012) -[2023-10-15 16:09:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 69763072. Throughput: 0: 1778.5, 1: 1780.7. Samples: 17452232. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:09:28,441][51532] Avg episode reward: [(0, '37.330'), (1, '38.440')] -[2023-10-15 16:09:29,308][52833] Updated weights for policy 0, policy_version 34020 (0.0007) -[2023-10-15 16:09:29,672][52833] Updated weights for policy 0, policy_version 34030 (0.0009) -[2023-10-15 16:09:30,032][52833] Updated weights for policy 0, policy_version 34040 (0.0008) -[2023-10-15 16:09:30,281][52866] Updated weights for policy 1, policy_version 34120 (0.0009) -[2023-10-15 16:09:30,659][52866] Updated weights for policy 1, policy_version 34130 (0.0009) -[2023-10-15 16:09:31,019][52866] Updated weights for policy 1, policy_version 34140 (0.0010) -[2023-10-15 16:09:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 69828608. Throughput: 0: 1780.9, 1: 1788.8. Samples: 17462106. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:09:33,442][51532] Avg episode reward: [(0, '38.340'), (1, '39.410')] -[2023-10-15 16:09:33,958][52833] Updated weights for policy 0, policy_version 34050 (0.0009) -[2023-10-15 16:09:34,368][52833] Updated weights for policy 0, policy_version 34060 (0.0009) -[2023-10-15 16:09:34,633][52866] Updated weights for policy 1, policy_version 34150 (0.0009) -[2023-10-15 16:09:34,735][52833] Updated weights for policy 0, policy_version 34070 (0.0008) -[2023-10-15 16:09:34,989][52866] Updated weights for policy 1, policy_version 34160 (0.0008) -[2023-10-15 16:09:35,111][52833] Updated weights for policy 0, policy_version 34080 (0.0009) -[2023-10-15 16:09:35,350][52866] Updated weights for policy 1, policy_version 34170 (0.0007) -[2023-10-15 16:09:38,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 69894144. Throughput: 0: 1782.4, 1: 1788.0. Samples: 17483996. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 16:09:38,442][51532] Avg episode reward: [(0, '38.210'), (1, '40.710')] -[2023-10-15 16:09:38,897][52833] Updated weights for policy 0, policy_version 34090 (0.0008) -[2023-10-15 16:09:39,269][52833] Updated weights for policy 0, policy_version 34100 (0.0008) -[2023-10-15 16:09:39,306][52866] Updated weights for policy 1, policy_version 34180 (0.0008) -[2023-10-15 16:09:39,637][52833] Updated weights for policy 0, policy_version 34110 (0.0009) -[2023-10-15 16:09:39,671][52866] Updated weights for policy 1, policy_version 34190 (0.0008) -[2023-10-15 16:09:40,045][52866] Updated weights for policy 1, policy_version 34200 (0.0010) -[2023-10-15 16:09:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 69959680. Throughput: 0: 1801.3, 1: 1782.7. Samples: 17506348. Policy #0 lag: (min: 10.0, avg: 18.1, max: 42.0) -[2023-10-15 16:09:43,443][51532] Avg episode reward: [(0, '36.640'), (1, '40.440')] -[2023-10-15 16:09:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000034208_35028992.pth... -[2023-10-15 16:09:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000032544_33325056.pth -[2023-10-15 16:09:43,528][52833] Updated weights for policy 0, policy_version 34120 (0.0008) -[2023-10-15 16:09:43,752][52866] Updated weights for policy 1, policy_version 34210 (0.0008) -[2023-10-15 16:09:43,898][52833] Updated weights for policy 0, policy_version 34130 (0.0010) -[2023-10-15 16:09:44,118][52866] Updated weights for policy 1, policy_version 34220 (0.0009) -[2023-10-15 16:09:44,273][52833] Updated weights for policy 0, policy_version 34140 (0.0008) -[2023-10-15 16:09:44,413][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000034144_34963456.pth... -[2023-10-15 16:09:44,442][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000032448_33226752.pth -[2023-10-15 16:09:44,479][52866] Updated weights for policy 1, policy_version 34230 (0.0008) -[2023-10-15 16:09:44,840][52866] Updated weights for policy 1, policy_version 34240 (0.0009) -[2023-10-15 16:09:48,203][52833] Updated weights for policy 0, policy_version 34150 (0.0008) -[2023-10-15 16:09:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 70025216. Throughput: 0: 1782.9, 1: 1780.2. Samples: 17516198. Policy #0 lag: (min: 10.0, avg: 18.1, max: 42.0) -[2023-10-15 16:09:48,442][51532] Avg episode reward: [(0, '36.610'), (1, '40.740')] -[2023-10-15 16:09:48,576][52833] Updated weights for policy 0, policy_version 34160 (0.0007) -[2023-10-15 16:09:48,617][52866] Updated weights for policy 1, policy_version 34250 (0.0010) -[2023-10-15 16:09:48,947][52833] Updated weights for policy 0, policy_version 34170 (0.0007) -[2023-10-15 16:09:48,973][52866] Updated weights for policy 1, policy_version 34260 (0.0007) -[2023-10-15 16:09:49,343][52866] Updated weights for policy 1, policy_version 34270 (0.0008) -[2023-10-15 16:09:52,648][52833] Updated weights for policy 0, policy_version 34180 (0.0009) -[2023-10-15 16:09:53,013][52833] Updated weights for policy 0, policy_version 34190 (0.0007) -[2023-10-15 16:09:53,204][52866] Updated weights for policy 1, policy_version 34280 (0.0009) -[2023-10-15 16:09:53,389][52833] Updated weights for policy 0, policy_version 34200 (0.0008) -[2023-10-15 16:09:53,441][51532] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 70090752. Throughput: 0: 1789.5, 1: 1778.4. Samples: 17538272. Policy #0 lag: (min: 10.0, avg: 18.1, max: 42.0) -[2023-10-15 16:09:53,441][51532] Avg episode reward: [(0, '36.370'), (1, '39.250')] -[2023-10-15 16:09:53,560][52866] Updated weights for policy 1, policy_version 34290 (0.0007) -[2023-10-15 16:09:53,925][52866] Updated weights for policy 1, policy_version 34300 (0.0008) -[2023-10-15 16:09:57,132][52833] Updated weights for policy 0, policy_version 34210 (0.0009) -[2023-10-15 16:09:57,505][52833] Updated weights for policy 0, policy_version 34220 (0.0009) -[2023-10-15 16:09:57,646][52866] Updated weights for policy 1, policy_version 34310 (0.0009) -[2023-10-15 16:09:57,864][52833] Updated weights for policy 0, policy_version 34230 (0.0008) -[2023-10-15 16:09:58,011][52866] Updated weights for policy 1, policy_version 34320 (0.0009) -[2023-10-15 16:09:58,234][52833] Updated weights for policy 0, policy_version 34240 (0.0007) -[2023-10-15 16:09:58,378][52866] Updated weights for policy 1, policy_version 34330 (0.0009) -[2023-10-15 16:09:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 70189056. Throughput: 0: 1788.8, 1: 1797.2. Samples: 17559310. Policy #0 lag: (min: 10.0, avg: 18.1, max: 42.0) -[2023-10-15 16:09:58,442][51532] Avg episode reward: [(0, '35.610'), (1, '38.060')] -[2023-10-15 16:10:01,907][52833] Updated weights for policy 0, policy_version 34250 (0.0007) -[2023-10-15 16:10:02,141][52866] Updated weights for policy 1, policy_version 34340 (0.0008) -[2023-10-15 16:10:02,276][52833] Updated weights for policy 0, policy_version 34260 (0.0007) -[2023-10-15 16:10:02,512][52866] Updated weights for policy 1, policy_version 34350 (0.0008) -[2023-10-15 16:10:02,641][52833] Updated weights for policy 0, policy_version 34270 (0.0008) -[2023-10-15 16:10:02,871][52866] Updated weights for policy 1, policy_version 34360 (0.0008) -[2023-10-15 16:10:03,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 70287360. Throughput: 0: 1788.5, 1: 1781.1. Samples: 17570680. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-15 16:10:03,441][51532] Avg episode reward: [(0, '37.960'), (1, '38.820')] -[2023-10-15 16:10:06,368][52833] Updated weights for policy 0, policy_version 34280 (0.0007) -[2023-10-15 16:10:06,674][52866] Updated weights for policy 1, policy_version 34370 (0.0009) -[2023-10-15 16:10:06,733][52833] Updated weights for policy 0, policy_version 34290 (0.0009) -[2023-10-15 16:10:07,046][52866] Updated weights for policy 1, policy_version 34380 (0.0009) -[2023-10-15 16:10:07,108][52833] Updated weights for policy 0, policy_version 34300 (0.0008) -[2023-10-15 16:10:07,405][52866] Updated weights for policy 1, policy_version 34390 (0.0007) -[2023-10-15 16:10:07,775][52866] Updated weights for policy 1, policy_version 34400 (0.0008) -[2023-10-15 16:10:08,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 70352896. Throughput: 0: 1788.2, 1: 1800.4. Samples: 17591544. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-15 16:10:08,441][51532] Avg episode reward: [(0, '36.710'), (1, '40.290')] -[2023-10-15 16:10:10,735][52833] Updated weights for policy 0, policy_version 34310 (0.0009) -[2023-10-15 16:10:11,103][52833] Updated weights for policy 0, policy_version 34320 (0.0010) -[2023-10-15 16:10:11,476][52833] Updated weights for policy 0, policy_version 34330 (0.0008) -[2023-10-15 16:10:11,478][52866] Updated weights for policy 1, policy_version 34410 (0.0009) -[2023-10-15 16:10:11,842][52866] Updated weights for policy 1, policy_version 34420 (0.0008) -[2023-10-15 16:10:12,209][52866] Updated weights for policy 1, policy_version 34430 (0.0008) -[2023-10-15 16:10:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 70418432. Throughput: 0: 1782.0, 1: 1783.1. Samples: 17612664. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-15 16:10:13,442][51532] Avg episode reward: [(0, '36.040'), (1, '37.760')] -[2023-10-15 16:10:15,295][52833] Updated weights for policy 0, policy_version 34340 (0.0010) -[2023-10-15 16:10:15,660][52833] Updated weights for policy 0, policy_version 34350 (0.0011) -[2023-10-15 16:10:16,031][52833] Updated weights for policy 0, policy_version 34360 (0.0009) -[2023-10-15 16:10:16,121][52866] Updated weights for policy 1, policy_version 34440 (0.0009) -[2023-10-15 16:10:16,490][52866] Updated weights for policy 1, policy_version 34450 (0.0008) -[2023-10-15 16:10:16,860][52866] Updated weights for policy 1, policy_version 34460 (0.0010) -[2023-10-15 16:10:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 70483968. Throughput: 0: 1794.6, 1: 1803.4. Samples: 17624014. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-15 16:10:18,442][51532] Avg episode reward: [(0, '37.430'), (1, '37.870')] -[2023-10-15 16:10:19,649][52833] Updated weights for policy 0, policy_version 34370 (0.0007) -[2023-10-15 16:10:20,015][52833] Updated weights for policy 0, policy_version 34380 (0.0007) -[2023-10-15 16:10:20,391][52833] Updated weights for policy 0, policy_version 34390 (0.0008) -[2023-10-15 16:10:20,471][52866] Updated weights for policy 1, policy_version 34470 (0.0008) -[2023-10-15 16:10:20,754][52833] Updated weights for policy 0, policy_version 34400 (0.0009) -[2023-10-15 16:10:20,840][52866] Updated weights for policy 1, policy_version 34480 (0.0009) -[2023-10-15 16:10:21,209][52866] Updated weights for policy 1, policy_version 34490 (0.0010) -[2023-10-15 16:10:23,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 70549504. Throughput: 0: 1792.0, 1: 1786.1. Samples: 17645012. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-15 16:10:23,441][51532] Avg episode reward: [(0, '39.490'), (1, '38.510')] -[2023-10-15 16:10:24,548][52833] Updated weights for policy 0, policy_version 34410 (0.0009) -[2023-10-15 16:10:24,905][52866] Updated weights for policy 1, policy_version 34500 (0.0010) -[2023-10-15 16:10:24,913][52833] Updated weights for policy 0, policy_version 34420 (0.0008) -[2023-10-15 16:10:25,261][52866] Updated weights for policy 1, policy_version 34510 (0.0009) -[2023-10-15 16:10:25,288][52833] Updated weights for policy 0, policy_version 34430 (0.0008) -[2023-10-15 16:10:25,628][52866] Updated weights for policy 1, policy_version 34520 (0.0009) -[2023-10-15 16:10:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 70615040. Throughput: 0: 1789.4, 1: 1781.2. Samples: 17667024. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-15 16:10:28,442][51532] Avg episode reward: [(0, '37.690'), (1, '38.370')] -[2023-10-15 16:10:29,068][52833] Updated weights for policy 0, policy_version 34440 (0.0008) -[2023-10-15 16:10:29,446][52833] Updated weights for policy 0, policy_version 34450 (0.0009) -[2023-10-15 16:10:29,593][52866] Updated weights for policy 1, policy_version 34530 (0.0009) -[2023-10-15 16:10:29,817][52833] Updated weights for policy 0, policy_version 34460 (0.0009) -[2023-10-15 16:10:29,972][52866] Updated weights for policy 1, policy_version 34540 (0.0008) -[2023-10-15 16:10:30,341][52866] Updated weights for policy 1, policy_version 34550 (0.0009) -[2023-10-15 16:10:30,701][52866] Updated weights for policy 1, policy_version 34560 (0.0010) -[2023-10-15 16:10:33,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 70680576. Throughput: 0: 1787.1, 1: 1781.5. Samples: 17676786. Policy #0 lag: (min: 20.0, avg: 30.6, max: 52.0) -[2023-10-15 16:10:33,442][51532] Avg episode reward: [(0, '37.740'), (1, '39.120')] -[2023-10-15 16:10:33,605][52833] Updated weights for policy 0, policy_version 34470 (0.0009) -[2023-10-15 16:10:33,968][52833] Updated weights for policy 0, policy_version 34480 (0.0010) -[2023-10-15 16:10:34,339][52833] Updated weights for policy 0, policy_version 34490 (0.0008) -[2023-10-15 16:10:34,433][52866] Updated weights for policy 1, policy_version 34570 (0.0007) -[2023-10-15 16:10:34,801][52866] Updated weights for policy 1, policy_version 34580 (0.0007) -[2023-10-15 16:10:35,174][52866] Updated weights for policy 1, policy_version 34590 (0.0012) -[2023-10-15 16:10:38,059][52833] Updated weights for policy 0, policy_version 34500 (0.0010) -[2023-10-15 16:10:38,425][52833] Updated weights for policy 0, policy_version 34510 (0.0009) -[2023-10-15 16:10:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 70746112. Throughput: 0: 1791.7, 1: 1781.3. Samples: 17699060. Policy #0 lag: (min: 20.0, avg: 30.6, max: 52.0) -[2023-10-15 16:10:38,441][51532] Avg episode reward: [(0, '38.680'), (1, '39.170')] -[2023-10-15 16:10:38,797][52833] Updated weights for policy 0, policy_version 34520 (0.0007) -[2023-10-15 16:10:38,899][52866] Updated weights for policy 1, policy_version 34600 (0.0009) -[2023-10-15 16:10:39,277][52866] Updated weights for policy 1, policy_version 34610 (0.0010) -[2023-10-15 16:10:39,637][52866] Updated weights for policy 1, policy_version 34620 (0.0008) -[2023-10-15 16:10:42,655][52833] Updated weights for policy 0, policy_version 34530 (0.0008) -[2023-10-15 16:10:43,018][52833] Updated weights for policy 0, policy_version 34540 (0.0008) -[2023-10-15 16:10:43,382][52866] Updated weights for policy 1, policy_version 34630 (0.0008) -[2023-10-15 16:10:43,394][52833] Updated weights for policy 0, policy_version 34550 (0.0010) -[2023-10-15 16:10:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 70811648. Throughput: 0: 1803.9, 1: 1793.9. Samples: 17721214. Policy #0 lag: (min: 20.0, avg: 30.6, max: 52.0) -[2023-10-15 16:10:43,442][51532] Avg episode reward: [(0, '37.540'), (1, '39.830')] -[2023-10-15 16:10:43,753][52866] Updated weights for policy 1, policy_version 34640 (0.0007) -[2023-10-15 16:10:43,762][52833] Updated weights for policy 0, policy_version 34560 (0.0009) -[2023-10-15 16:10:44,112][52866] Updated weights for policy 1, policy_version 34650 (0.0007) -[2023-10-15 16:10:47,529][52833] Updated weights for policy 0, policy_version 34570 (0.0008) -[2023-10-15 16:10:47,875][52866] Updated weights for policy 1, policy_version 34660 (0.0009) -[2023-10-15 16:10:47,899][52833] Updated weights for policy 0, policy_version 34580 (0.0008) -[2023-10-15 16:10:48,237][52866] Updated weights for policy 1, policy_version 34670 (0.0008) -[2023-10-15 16:10:48,270][52833] Updated weights for policy 0, policy_version 34590 (0.0008) -[2023-10-15 16:10:48,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 70909952. Throughput: 0: 1793.1, 1: 1783.2. Samples: 17731614. Policy #0 lag: (min: 20.0, avg: 30.6, max: 52.0) -[2023-10-15 16:10:48,441][51532] Avg episode reward: [(0, '40.050'), (1, '39.070')] -[2023-10-15 16:10:48,602][52866] Updated weights for policy 1, policy_version 34680 (0.0008) -[2023-10-15 16:10:51,964][52833] Updated weights for policy 0, policy_version 34600 (0.0007) -[2023-10-15 16:10:52,337][52833] Updated weights for policy 0, policy_version 34610 (0.0008) -[2023-10-15 16:10:52,352][52866] Updated weights for policy 1, policy_version 34690 (0.0008) -[2023-10-15 16:10:52,713][52833] Updated weights for policy 0, policy_version 34620 (0.0008) -[2023-10-15 16:10:52,721][52866] Updated weights for policy 1, policy_version 34700 (0.0009) -[2023-10-15 16:10:53,078][52866] Updated weights for policy 1, policy_version 34710 (0.0008) -[2023-10-15 16:10:53,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 70975488. Throughput: 0: 1806.9, 1: 1793.4. Samples: 17753558. Policy #0 lag: (min: 18.0, avg: 24.4, max: 50.0) -[2023-10-15 16:10:53,442][51532] Avg episode reward: [(0, '38.800'), (1, '40.080')] -[2023-10-15 16:10:53,446][52866] Updated weights for policy 1, policy_version 34720 (0.0008) -[2023-10-15 16:10:56,392][52833] Updated weights for policy 0, policy_version 34630 (0.0010) -[2023-10-15 16:10:56,764][52833] Updated weights for policy 0, policy_version 34640 (0.0009) -[2023-10-15 16:10:57,134][52833] Updated weights for policy 0, policy_version 34650 (0.0008) -[2023-10-15 16:10:57,139][52866] Updated weights for policy 1, policy_version 34730 (0.0008) -[2023-10-15 16:10:57,505][52866] Updated weights for policy 1, policy_version 34740 (0.0008) -[2023-10-15 16:10:57,868][52866] Updated weights for policy 1, policy_version 34750 (0.0008) -[2023-10-15 16:10:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 71073792. Throughput: 0: 1790.2, 1: 1783.5. Samples: 17773480. Policy #0 lag: (min: 18.0, avg: 24.4, max: 50.0) -[2023-10-15 16:10:58,441][51532] Avg episode reward: [(0, '39.990'), (1, '40.840')] -[2023-10-15 16:11:00,962][52833] Updated weights for policy 0, policy_version 34660 (0.0008) -[2023-10-15 16:11:01,328][52833] Updated weights for policy 0, policy_version 34670 (0.0009) -[2023-10-15 16:11:01,694][52833] Updated weights for policy 0, policy_version 34680 (0.0009) -[2023-10-15 16:11:01,727][52866] Updated weights for policy 1, policy_version 34760 (0.0009) -[2023-10-15 16:11:02,086][52866] Updated weights for policy 1, policy_version 34770 (0.0008) -[2023-10-15 16:11:02,455][52866] Updated weights for policy 1, policy_version 34780 (0.0008) -[2023-10-15 16:11:03,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 71139328. Throughput: 0: 1811.2, 1: 1790.8. Samples: 17786100. Policy #0 lag: (min: 18.0, avg: 24.4, max: 50.0) -[2023-10-15 16:11:03,442][51532] Avg episode reward: [(0, '40.350'), (1, '38.480')] -[2023-10-15 16:11:05,449][52833] Updated weights for policy 0, policy_version 34690 (0.0008) -[2023-10-15 16:11:05,818][52833] Updated weights for policy 0, policy_version 34700 (0.0011) -[2023-10-15 16:11:06,187][52833] Updated weights for policy 0, policy_version 34710 (0.0008) -[2023-10-15 16:11:06,244][52866] Updated weights for policy 1, policy_version 34790 (0.0007) -[2023-10-15 16:11:06,554][52833] Updated weights for policy 0, policy_version 34720 (0.0008) -[2023-10-15 16:11:06,599][52866] Updated weights for policy 1, policy_version 34800 (0.0009) -[2023-10-15 16:11:06,966][52866] Updated weights for policy 1, policy_version 34810 (0.0011) -[2023-10-15 16:11:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 71204864. Throughput: 0: 1789.1, 1: 1789.1. Samples: 17806032. Policy #0 lag: (min: 18.0, avg: 24.4, max: 50.0) -[2023-10-15 16:11:08,442][51532] Avg episode reward: [(0, '40.120'), (1, '40.220')] -[2023-10-15 16:11:10,429][52833] Updated weights for policy 0, policy_version 34730 (0.0008) -[2023-10-15 16:11:10,651][52866] Updated weights for policy 1, policy_version 34820 (0.0009) -[2023-10-15 16:11:10,792][52833] Updated weights for policy 0, policy_version 34740 (0.0009) -[2023-10-15 16:11:11,017][52866] Updated weights for policy 1, policy_version 34830 (0.0007) -[2023-10-15 16:11:11,160][52833] Updated weights for policy 0, policy_version 34750 (0.0007) -[2023-10-15 16:11:11,387][52866] Updated weights for policy 1, policy_version 34840 (0.0009) -[2023-10-15 16:11:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71270400. Throughput: 0: 1789.4, 1: 1792.6. Samples: 17828214. Policy #0 lag: (min: 18.0, avg: 24.4, max: 50.0) -[2023-10-15 16:11:13,442][51532] Avg episode reward: [(0, '41.420'), (1, '38.920')] -[2023-10-15 16:11:14,851][52833] Updated weights for policy 0, policy_version 34760 (0.0010) -[2023-10-15 16:11:15,218][52833] Updated weights for policy 0, policy_version 34770 (0.0009) -[2023-10-15 16:11:15,224][52866] Updated weights for policy 1, policy_version 34850 (0.0008) -[2023-10-15 16:11:15,588][52866] Updated weights for policy 1, policy_version 34860 (0.0009) -[2023-10-15 16:11:15,593][52833] Updated weights for policy 0, policy_version 34780 (0.0008) -[2023-10-15 16:11:15,950][52866] Updated weights for policy 1, policy_version 34870 (0.0007) -[2023-10-15 16:11:16,310][52866] Updated weights for policy 1, policy_version 34880 (0.0007) -[2023-10-15 16:11:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 71335936. Throughput: 0: 1789.1, 1: 1797.7. Samples: 17838190. Policy #0 lag: (min: 27.0, avg: 27.2, max: 37.0) -[2023-10-15 16:11:18,442][51532] Avg episode reward: [(0, '41.050'), (1, '38.430')] -[2023-10-15 16:11:19,253][52833] Updated weights for policy 0, policy_version 34790 (0.0008) -[2023-10-15 16:11:19,625][52833] Updated weights for policy 0, policy_version 34800 (0.0008) -[2023-10-15 16:11:19,989][52833] Updated weights for policy 0, policy_version 34810 (0.0007) -[2023-10-15 16:11:20,075][52866] Updated weights for policy 1, policy_version 34890 (0.0008) -[2023-10-15 16:11:20,451][52866] Updated weights for policy 1, policy_version 34900 (0.0007) -[2023-10-15 16:11:20,811][52866] Updated weights for policy 1, policy_version 34910 (0.0007) -[2023-10-15 16:11:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 71401472. Throughput: 0: 1786.2, 1: 1792.1. Samples: 17860084. Policy #0 lag: (min: 27.0, avg: 27.2, max: 37.0) -[2023-10-15 16:11:23,442][51532] Avg episode reward: [(0, '42.260'), (1, '37.700')] -[2023-10-15 16:11:23,804][52833] Updated weights for policy 0, policy_version 34820 (0.0008) -[2023-10-15 16:11:24,176][52833] Updated weights for policy 0, policy_version 34830 (0.0008) -[2023-10-15 16:11:24,546][52833] Updated weights for policy 0, policy_version 34840 (0.0009) -[2023-10-15 16:11:24,658][52866] Updated weights for policy 1, policy_version 34920 (0.0009) -[2023-10-15 16:11:25,013][52866] Updated weights for policy 1, policy_version 34930 (0.0008) -[2023-10-15 16:11:25,385][52866] Updated weights for policy 1, policy_version 34940 (0.0009) -[2023-10-15 16:11:28,159][52833] Updated weights for policy 0, policy_version 34850 (0.0008) -[2023-10-15 16:11:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 71467008. Throughput: 0: 1798.1, 1: 1789.5. Samples: 17882656. Policy #0 lag: (min: 27.0, avg: 27.2, max: 37.0) -[2023-10-15 16:11:28,441][51532] Avg episode reward: [(0, '42.490'), (1, '38.150')] -[2023-10-15 16:11:28,531][52833] Updated weights for policy 0, policy_version 34860 (0.0007) -[2023-10-15 16:11:28,894][52833] Updated weights for policy 0, policy_version 34870 (0.0009) -[2023-10-15 16:11:29,211][52866] Updated weights for policy 1, policy_version 34950 (0.0008) -[2023-10-15 16:11:29,264][52833] Updated weights for policy 0, policy_version 34880 (0.0007) -[2023-10-15 16:11:29,580][52866] Updated weights for policy 1, policy_version 34960 (0.0011) -[2023-10-15 16:11:29,953][52866] Updated weights for policy 1, policy_version 34970 (0.0010) -[2023-10-15 16:11:33,089][52833] Updated weights for policy 0, policy_version 34890 (0.0007) -[2023-10-15 16:11:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14218.0). Total num frames: 71532544. Throughput: 0: 1786.3, 1: 1787.6. Samples: 17892442. Policy #0 lag: (min: 27.0, avg: 27.2, max: 37.0) -[2023-10-15 16:11:33,441][51532] Avg episode reward: [(0, '42.410'), (1, '39.100')] -[2023-10-15 16:11:33,457][52833] Updated weights for policy 0, policy_version 34900 (0.0007) -[2023-10-15 16:11:33,546][52866] Updated weights for policy 1, policy_version 34980 (0.0009) -[2023-10-15 16:11:33,824][52833] Updated weights for policy 0, policy_version 34910 (0.0007) -[2023-10-15 16:11:33,913][52866] Updated weights for policy 1, policy_version 34990 (0.0010) -[2023-10-15 16:11:34,280][52866] Updated weights for policy 1, policy_version 35000 (0.0010) -[2023-10-15 16:11:37,496][52833] Updated weights for policy 0, policy_version 34920 (0.0008) -[2023-10-15 16:11:37,858][52833] Updated weights for policy 0, policy_version 34930 (0.0007) -[2023-10-15 16:11:37,878][52866] Updated weights for policy 1, policy_version 35010 (0.0007) -[2023-10-15 16:11:38,226][52833] Updated weights for policy 0, policy_version 34940 (0.0007) -[2023-10-15 16:11:38,245][52866] Updated weights for policy 1, policy_version 35020 (0.0008) -[2023-10-15 16:11:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 71630848. Throughput: 0: 1794.7, 1: 1795.8. Samples: 17915130. Policy #0 lag: (min: 27.0, avg: 27.2, max: 37.0) -[2023-10-15 16:11:38,441][51532] Avg episode reward: [(0, '41.530'), (1, '40.390')] -[2023-10-15 16:11:38,624][52866] Updated weights for policy 1, policy_version 35030 (0.0009) -[2023-10-15 16:11:38,997][52866] Updated weights for policy 1, policy_version 35040 (0.0009) -[2023-10-15 16:11:42,117][52833] Updated weights for policy 0, policy_version 34950 (0.0008) -[2023-10-15 16:11:42,493][52833] Updated weights for policy 0, policy_version 34960 (0.0007) -[2023-10-15 16:11:42,840][52866] Updated weights for policy 1, policy_version 35050 (0.0007) -[2023-10-15 16:11:42,869][52833] Updated weights for policy 0, policy_version 34970 (0.0007) -[2023-10-15 16:11:43,212][52866] Updated weights for policy 1, policy_version 35060 (0.0008) -[2023-10-15 16:11:43,441][51532] Fps is (10 sec: 16383.1, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 71696384. Throughput: 0: 1796.5, 1: 1809.0. Samples: 17935730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:11:43,442][51532] Avg episode reward: [(0, '41.750'), (1, '41.020')] -[2023-10-15 16:11:43,455][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000034976_35815424.pth... -[2023-10-15 16:11:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000033280_34078720.pth -[2023-10-15 16:11:43,579][52866] Updated weights for policy 1, policy_version 35070 (0.0011) -[2023-10-15 16:11:43,649][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000035072_35913728.pth... -[2023-10-15 16:11:43,678][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000033376_34177024.pth -[2023-10-15 16:11:46,523][52833] Updated weights for policy 0, policy_version 34980 (0.0008) -[2023-10-15 16:11:46,899][52833] Updated weights for policy 0, policy_version 34990 (0.0008) -[2023-10-15 16:11:47,278][52833] Updated weights for policy 0, policy_version 35000 (0.0007) -[2023-10-15 16:11:47,289][52866] Updated weights for policy 1, policy_version 35080 (0.0009) -[2023-10-15 16:11:47,654][52866] Updated weights for policy 1, policy_version 35090 (0.0007) -[2023-10-15 16:11:48,028][52866] Updated weights for policy 1, policy_version 35100 (0.0009) -[2023-10-15 16:11:48,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 71794688. Throughput: 0: 1788.6, 1: 1794.6. Samples: 17947342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:11:48,442][51532] Avg episode reward: [(0, '41.210'), (1, '38.240')] -[2023-10-15 16:11:51,009][52833] Updated weights for policy 0, policy_version 35010 (0.0009) -[2023-10-15 16:11:51,384][52833] Updated weights for policy 0, policy_version 35020 (0.0009) -[2023-10-15 16:11:51,744][52833] Updated weights for policy 0, policy_version 35030 (0.0008) -[2023-10-15 16:11:51,811][52866] Updated weights for policy 1, policy_version 35110 (0.0008) -[2023-10-15 16:11:52,110][52833] Updated weights for policy 0, policy_version 35040 (0.0007) -[2023-10-15 16:11:52,173][52866] Updated weights for policy 1, policy_version 35120 (0.0009) -[2023-10-15 16:11:52,545][52866] Updated weights for policy 1, policy_version 35130 (0.0008) -[2023-10-15 16:11:53,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 71860224. Throughput: 0: 1794.3, 1: 1804.6. Samples: 17967982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:11:53,442][51532] Avg episode reward: [(0, '40.880'), (1, '38.290')] -[2023-10-15 16:11:55,914][52833] Updated weights for policy 0, policy_version 35050 (0.0008) -[2023-10-15 16:11:56,211][52866] Updated weights for policy 1, policy_version 35140 (0.0007) -[2023-10-15 16:11:56,282][52833] Updated weights for policy 0, policy_version 35060 (0.0007) -[2023-10-15 16:11:56,575][52866] Updated weights for policy 1, policy_version 35150 (0.0009) -[2023-10-15 16:11:56,639][52833] Updated weights for policy 0, policy_version 35070 (0.0008) -[2023-10-15 16:11:56,938][52866] Updated weights for policy 1, policy_version 35160 (0.0008) -[2023-10-15 16:11:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 71925760. Throughput: 0: 1782.0, 1: 1788.8. Samples: 17988898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:11:58,442][51532] Avg episode reward: [(0, '39.570'), (1, '39.640')] -[2023-10-15 16:12:00,502][52833] Updated weights for policy 0, policy_version 35080 (0.0009) -[2023-10-15 16:12:00,737][52866] Updated weights for policy 1, policy_version 35170 (0.0008) -[2023-10-15 16:12:00,871][52833] Updated weights for policy 0, policy_version 35090 (0.0009) -[2023-10-15 16:12:01,111][52866] Updated weights for policy 1, policy_version 35180 (0.0008) -[2023-10-15 16:12:01,240][52833] Updated weights for policy 0, policy_version 35100 (0.0008) -[2023-10-15 16:12:01,488][52866] Updated weights for policy 1, policy_version 35190 (0.0007) -[2023-10-15 16:12:01,851][52866] Updated weights for policy 1, policy_version 35200 (0.0007) -[2023-10-15 16:12:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71991296. Throughput: 0: 1798.8, 1: 1804.7. Samples: 18000348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:12:03,442][51532] Avg episode reward: [(0, '38.540'), (1, '39.630')] -[2023-10-15 16:12:05,023][52833] Updated weights for policy 0, policy_version 35110 (0.0009) -[2023-10-15 16:12:05,391][52833] Updated weights for policy 0, policy_version 35120 (0.0008) -[2023-10-15 16:12:05,682][52866] Updated weights for policy 1, policy_version 35210 (0.0007) -[2023-10-15 16:12:05,768][52833] Updated weights for policy 0, policy_version 35130 (0.0007) -[2023-10-15 16:12:06,051][52866] Updated weights for policy 1, policy_version 35220 (0.0008) -[2023-10-15 16:12:06,421][52866] Updated weights for policy 1, policy_version 35230 (0.0009) -[2023-10-15 16:12:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 72056832. Throughput: 0: 1785.5, 1: 1787.3. Samples: 18020860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:12:08,442][51532] Avg episode reward: [(0, '39.150'), (1, '40.530')] -[2023-10-15 16:12:09,584][52833] Updated weights for policy 0, policy_version 35140 (0.0008) -[2023-10-15 16:12:09,952][52833] Updated weights for policy 0, policy_version 35150 (0.0007) -[2023-10-15 16:12:10,128][52866] Updated weights for policy 1, policy_version 35240 (0.0008) -[2023-10-15 16:12:10,325][52833] Updated weights for policy 0, policy_version 35160 (0.0009) -[2023-10-15 16:12:10,489][52866] Updated weights for policy 1, policy_version 35250 (0.0008) -[2023-10-15 16:12:10,864][52866] Updated weights for policy 1, policy_version 35260 (0.0010) -[2023-10-15 16:12:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 72122368. Throughput: 0: 1778.3, 1: 1791.1. Samples: 18043276. Policy #0 lag: (min: 27.0, avg: 46.2, max: 48.0) -[2023-10-15 16:12:13,442][51532] Avg episode reward: [(0, '39.720'), (1, '39.520')] -[2023-10-15 16:12:14,040][52833] Updated weights for policy 0, policy_version 35170 (0.0009) -[2023-10-15 16:12:14,408][52833] Updated weights for policy 0, policy_version 35180 (0.0010) -[2023-10-15 16:12:14,724][52866] Updated weights for policy 1, policy_version 35270 (0.0009) -[2023-10-15 16:12:14,786][52833] Updated weights for policy 0, policy_version 35190 (0.0007) -[2023-10-15 16:12:15,090][52866] Updated weights for policy 1, policy_version 35280 (0.0009) -[2023-10-15 16:12:15,147][52833] Updated weights for policy 0, policy_version 35200 (0.0008) -[2023-10-15 16:12:15,448][52866] Updated weights for policy 1, policy_version 35290 (0.0008) -[2023-10-15 16:12:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 72187904. Throughput: 0: 1780.8, 1: 1787.0. Samples: 18052994. Policy #0 lag: (min: 27.0, avg: 46.2, max: 48.0) -[2023-10-15 16:12:18,441][51532] Avg episode reward: [(0, '40.390'), (1, '39.400')] -[2023-10-15 16:12:18,857][52833] Updated weights for policy 0, policy_version 35210 (0.0009) -[2023-10-15 16:12:19,228][52833] Updated weights for policy 0, policy_version 35220 (0.0008) -[2023-10-15 16:12:19,283][52866] Updated weights for policy 1, policy_version 35300 (0.0007) -[2023-10-15 16:12:19,592][52833] Updated weights for policy 0, policy_version 35230 (0.0007) -[2023-10-15 16:12:19,651][52866] Updated weights for policy 1, policy_version 35310 (0.0008) -[2023-10-15 16:12:20,019][52866] Updated weights for policy 1, policy_version 35320 (0.0008) -[2023-10-15 16:12:23,387][52833] Updated weights for policy 0, policy_version 35240 (0.0009) -[2023-10-15 16:12:23,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 72253440. Throughput: 0: 1778.2, 1: 1785.7. Samples: 18075504. Policy #0 lag: (min: 27.0, avg: 46.2, max: 48.0) -[2023-10-15 16:12:23,442][51532] Avg episode reward: [(0, '40.190'), (1, '40.560')] -[2023-10-15 16:12:23,467][52866] Updated weights for policy 1, policy_version 35330 (0.0008) -[2023-10-15 16:12:23,755][52833] Updated weights for policy 0, policy_version 35250 (0.0008) -[2023-10-15 16:12:23,835][52866] Updated weights for policy 1, policy_version 35340 (0.0008) -[2023-10-15 16:12:24,119][52833] Updated weights for policy 0, policy_version 35260 (0.0007) -[2023-10-15 16:12:24,202][52866] Updated weights for policy 1, policy_version 35350 (0.0008) -[2023-10-15 16:12:24,567][52866] Updated weights for policy 1, policy_version 35360 (0.0011) -[2023-10-15 16:12:27,978][52833] Updated weights for policy 0, policy_version 35270 (0.0010) -[2023-10-15 16:12:28,343][52833] Updated weights for policy 0, policy_version 35280 (0.0008) -[2023-10-15 16:12:28,353][52866] Updated weights for policy 1, policy_version 35370 (0.0008) -[2023-10-15 16:12:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 72318976. Throughput: 0: 1799.3, 1: 1803.0. Samples: 18097830. Policy #0 lag: (min: 27.0, avg: 46.2, max: 48.0) -[2023-10-15 16:12:28,441][51532] Avg episode reward: [(0, '38.550'), (1, '38.870')] -[2023-10-15 16:12:28,720][52833] Updated weights for policy 0, policy_version 35290 (0.0008) -[2023-10-15 16:12:28,724][52866] Updated weights for policy 1, policy_version 35380 (0.0007) -[2023-10-15 16:12:29,087][52866] Updated weights for policy 1, policy_version 35390 (0.0008) -[2023-10-15 16:12:32,529][52833] Updated weights for policy 0, policy_version 35300 (0.0009) -[2023-10-15 16:12:32,891][52833] Updated weights for policy 0, policy_version 35310 (0.0008) -[2023-10-15 16:12:32,975][52866] Updated weights for policy 1, policy_version 35400 (0.0008) -[2023-10-15 16:12:33,257][52833] Updated weights for policy 0, policy_version 35320 (0.0009) -[2023-10-15 16:12:33,333][52866] Updated weights for policy 1, policy_version 35410 (0.0008) -[2023-10-15 16:12:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 72384512. Throughput: 0: 1775.3, 1: 1789.8. Samples: 18107772. Policy #0 lag: (min: 27.0, avg: 46.2, max: 48.0) -[2023-10-15 16:12:33,442][51532] Avg episode reward: [(0, '39.170'), (1, '39.600')] -[2023-10-15 16:12:33,706][52866] Updated weights for policy 1, policy_version 35420 (0.0007) -[2023-10-15 16:12:37,123][52833] Updated weights for policy 0, policy_version 35330 (0.0008) -[2023-10-15 16:12:37,467][52866] Updated weights for policy 1, policy_version 35430 (0.0008) -[2023-10-15 16:12:37,481][52833] Updated weights for policy 0, policy_version 35340 (0.0008) -[2023-10-15 16:12:37,829][52866] Updated weights for policy 1, policy_version 35440 (0.0008) -[2023-10-15 16:12:37,855][52833] Updated weights for policy 0, policy_version 35350 (0.0008) -[2023-10-15 16:12:38,199][52866] Updated weights for policy 1, policy_version 35450 (0.0008) -[2023-10-15 16:12:38,222][52833] Updated weights for policy 0, policy_version 35360 (0.0007) -[2023-10-15 16:12:38,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 72515584. Throughput: 0: 1798.3, 1: 1802.8. Samples: 18130030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:12:38,442][51532] Avg episode reward: [(0, '39.170'), (1, '41.690')] -[2023-10-15 16:12:42,033][52866] Updated weights for policy 1, policy_version 35460 (0.0009) -[2023-10-15 16:12:42,066][52833] Updated weights for policy 0, policy_version 35370 (0.0008) -[2023-10-15 16:12:42,406][52866] Updated weights for policy 1, policy_version 35470 (0.0008) -[2023-10-15 16:12:42,431][52833] Updated weights for policy 0, policy_version 35380 (0.0008) -[2023-10-15 16:12:42,762][52866] Updated weights for policy 1, policy_version 35480 (0.0008) -[2023-10-15 16:12:42,809][52833] Updated weights for policy 0, policy_version 35390 (0.0007) -[2023-10-15 16:12:43,441][51532] Fps is (10 sec: 19660.6, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 72581120. Throughput: 0: 1781.7, 1: 1792.0. Samples: 18149714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:12:43,442][51532] Avg episode reward: [(0, '40.380'), (1, '41.310')] -[2023-10-15 16:12:46,481][52833] Updated weights for policy 0, policy_version 35400 (0.0009) -[2023-10-15 16:12:46,537][52866] Updated weights for policy 1, policy_version 35490 (0.0009) -[2023-10-15 16:12:46,844][52833] Updated weights for policy 0, policy_version 35410 (0.0007) -[2023-10-15 16:12:46,894][52866] Updated weights for policy 1, policy_version 35500 (0.0008) -[2023-10-15 16:12:47,213][52833] Updated weights for policy 0, policy_version 35420 (0.0007) -[2023-10-15 16:12:47,257][52866] Updated weights for policy 1, policy_version 35510 (0.0008) -[2023-10-15 16:12:47,632][52866] Updated weights for policy 1, policy_version 35520 (0.0009) -[2023-10-15 16:12:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 72646656. Throughput: 0: 1799.7, 1: 1797.7. Samples: 18162230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:12:48,441][51532] Avg episode reward: [(0, '39.150'), (1, '43.650')] -[2023-10-15 16:12:50,961][52833] Updated weights for policy 0, policy_version 35430 (0.0007) -[2023-10-15 16:12:51,327][52833] Updated weights for policy 0, policy_version 35440 (0.0008) -[2023-10-15 16:12:51,433][52866] Updated weights for policy 1, policy_version 35530 (0.0007) -[2023-10-15 16:12:51,691][52833] Updated weights for policy 0, policy_version 35450 (0.0007) -[2023-10-15 16:12:51,796][52866] Updated weights for policy 1, policy_version 35540 (0.0007) -[2023-10-15 16:12:52,160][52866] Updated weights for policy 1, policy_version 35550 (0.0009) -[2023-10-15 16:12:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 72712192. Throughput: 0: 1779.3, 1: 1801.3. Samples: 18181988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:12:53,442][51532] Avg episode reward: [(0, '42.250'), (1, '42.490')] -[2023-10-15 16:12:55,371][52833] Updated weights for policy 0, policy_version 35460 (0.0008) -[2023-10-15 16:12:55,735][52833] Updated weights for policy 0, policy_version 35470 (0.0007) -[2023-10-15 16:12:55,824][52866] Updated weights for policy 1, policy_version 35560 (0.0007) -[2023-10-15 16:12:56,103][52833] Updated weights for policy 0, policy_version 35480 (0.0007) -[2023-10-15 16:12:56,190][52866] Updated weights for policy 1, policy_version 35570 (0.0008) -[2023-10-15 16:12:56,560][52866] Updated weights for policy 1, policy_version 35580 (0.0007) -[2023-10-15 16:12:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 72777728. Throughput: 0: 1776.8, 1: 1792.7. Samples: 18203906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:12:58,442][51532] Avg episode reward: [(0, '39.600'), (1, '41.350')] -[2023-10-15 16:12:59,889][52833] Updated weights for policy 0, policy_version 35490 (0.0009) -[2023-10-15 16:13:00,081][52866] Updated weights for policy 1, policy_version 35590 (0.0008) -[2023-10-15 16:13:00,260][52833] Updated weights for policy 0, policy_version 35500 (0.0008) -[2023-10-15 16:13:00,444][52866] Updated weights for policy 1, policy_version 35600 (0.0007) -[2023-10-15 16:13:00,623][52833] Updated weights for policy 0, policy_version 35510 (0.0007) -[2023-10-15 16:13:00,819][52866] Updated weights for policy 1, policy_version 35610 (0.0009) -[2023-10-15 16:13:00,992][52833] Updated weights for policy 0, policy_version 35520 (0.0007) -[2023-10-15 16:13:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 72843264. Throughput: 0: 1779.8, 1: 1804.4. Samples: 18214284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:13:03,442][51532] Avg episode reward: [(0, '38.090'), (1, '42.320')] -[2023-10-15 16:13:04,586][52866] Updated weights for policy 1, policy_version 35620 (0.0009) -[2023-10-15 16:13:04,742][52833] Updated weights for policy 0, policy_version 35530 (0.0008) -[2023-10-15 16:13:04,954][52866] Updated weights for policy 1, policy_version 35630 (0.0008) -[2023-10-15 16:13:05,103][52833] Updated weights for policy 0, policy_version 35540 (0.0007) -[2023-10-15 16:13:05,315][52866] Updated weights for policy 1, policy_version 35640 (0.0008) -[2023-10-15 16:13:05,474][52833] Updated weights for policy 0, policy_version 35550 (0.0009) -[2023-10-15 16:13:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 72908800. Throughput: 0: 1770.8, 1: 1795.5. Samples: 18235988. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) -[2023-10-15 16:13:08,441][51532] Avg episode reward: [(0, '38.120'), (1, '41.270')] -[2023-10-15 16:13:09,039][52866] Updated weights for policy 1, policy_version 35650 (0.0010) -[2023-10-15 16:13:09,414][52866] Updated weights for policy 1, policy_version 35660 (0.0007) -[2023-10-15 16:13:09,460][52833] Updated weights for policy 0, policy_version 35560 (0.0009) -[2023-10-15 16:13:09,788][52866] Updated weights for policy 1, policy_version 35670 (0.0008) -[2023-10-15 16:13:09,825][52833] Updated weights for policy 0, policy_version 35570 (0.0008) -[2023-10-15 16:13:10,152][52866] Updated weights for policy 1, policy_version 35680 (0.0007) -[2023-10-15 16:13:10,201][52833] Updated weights for policy 0, policy_version 35580 (0.0010) -[2023-10-15 16:13:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 72974336. Throughput: 0: 1770.8, 1: 1797.6. Samples: 18258408. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) -[2023-10-15 16:13:13,442][51532] Avg episode reward: [(0, '38.070'), (1, '40.440')] -[2023-10-15 16:13:13,844][52866] Updated weights for policy 1, policy_version 35690 (0.0007) -[2023-10-15 16:13:14,061][52833] Updated weights for policy 0, policy_version 35590 (0.0010) -[2023-10-15 16:13:14,205][52866] Updated weights for policy 1, policy_version 35700 (0.0007) -[2023-10-15 16:13:14,433][52833] Updated weights for policy 0, policy_version 35600 (0.0008) -[2023-10-15 16:13:14,578][52866] Updated weights for policy 1, policy_version 35710 (0.0008) -[2023-10-15 16:13:14,801][52833] Updated weights for policy 0, policy_version 35610 (0.0008) -[2023-10-15 16:13:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 73039872. Throughput: 0: 1771.5, 1: 1794.9. Samples: 18268260. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) -[2023-10-15 16:13:18,441][51532] Avg episode reward: [(0, '40.220'), (1, '38.540')] -[2023-10-15 16:13:18,534][52866] Updated weights for policy 1, policy_version 35720 (0.0008) -[2023-10-15 16:13:18,619][52833] Updated weights for policy 0, policy_version 35620 (0.0010) -[2023-10-15 16:13:18,897][52866] Updated weights for policy 1, policy_version 35730 (0.0007) -[2023-10-15 16:13:18,981][52833] Updated weights for policy 0, policy_version 35630 (0.0009) -[2023-10-15 16:13:19,264][52866] Updated weights for policy 1, policy_version 35740 (0.0008) -[2023-10-15 16:13:19,351][52833] Updated weights for policy 0, policy_version 35640 (0.0007) -[2023-10-15 16:13:23,074][52866] Updated weights for policy 1, policy_version 35750 (0.0008) -[2023-10-15 16:13:23,122][52833] Updated weights for policy 0, policy_version 35650 (0.0009) -[2023-10-15 16:13:23,432][52866] Updated weights for policy 1, policy_version 35760 (0.0009) -[2023-10-15 16:13:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 73105408. Throughput: 0: 1768.4, 1: 1792.8. Samples: 18290286. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) -[2023-10-15 16:13:23,441][51532] Avg episode reward: [(0, '40.610'), (1, '38.670')] -[2023-10-15 16:13:23,485][52833] Updated weights for policy 0, policy_version 35660 (0.0008) -[2023-10-15 16:13:23,802][52866] Updated weights for policy 1, policy_version 35770 (0.0008) -[2023-10-15 16:13:23,851][52833] Updated weights for policy 0, policy_version 35670 (0.0007) -[2023-10-15 16:13:24,220][52833] Updated weights for policy 0, policy_version 35680 (0.0007) -[2023-10-15 16:13:27,579][52866] Updated weights for policy 1, policy_version 35780 (0.0008) -[2023-10-15 16:13:27,949][52866] Updated weights for policy 1, policy_version 35790 (0.0008) -[2023-10-15 16:13:28,070][52833] Updated weights for policy 0, policy_version 35690 (0.0008) -[2023-10-15 16:13:28,321][52866] Updated weights for policy 1, policy_version 35800 (0.0009) -[2023-10-15 16:13:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 73170944. Throughput: 0: 1792.1, 1: 1810.0. Samples: 18311804. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) -[2023-10-15 16:13:28,441][51532] Avg episode reward: [(0, '40.130'), (1, '41.400')] -[2023-10-15 16:13:28,445][52833] Updated weights for policy 0, policy_version 35700 (0.0008) -[2023-10-15 16:13:28,811][52833] Updated weights for policy 0, policy_version 35710 (0.0007) -[2023-10-15 16:13:32,070][52866] Updated weights for policy 1, policy_version 35810 (0.0008) -[2023-10-15 16:13:32,433][52866] Updated weights for policy 1, policy_version 35820 (0.0008) -[2023-10-15 16:13:32,639][52833] Updated weights for policy 0, policy_version 35720 (0.0007) -[2023-10-15 16:13:32,804][52866] Updated weights for policy 1, policy_version 35830 (0.0008) -[2023-10-15 16:13:33,014][52833] Updated weights for policy 0, policy_version 35730 (0.0008) -[2023-10-15 16:13:33,168][52866] Updated weights for policy 1, policy_version 35840 (0.0009) -[2023-10-15 16:13:33,383][52833] Updated weights for policy 0, policy_version 35740 (0.0010) -[2023-10-15 16:13:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 73269248. Throughput: 0: 1763.7, 1: 1797.6. Samples: 18322492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:13:33,441][51532] Avg episode reward: [(0, '40.030'), (1, '39.410')] -[2023-10-15 16:13:36,972][52866] Updated weights for policy 1, policy_version 35850 (0.0007) -[2023-10-15 16:13:37,178][52833] Updated weights for policy 0, policy_version 35750 (0.0008) -[2023-10-15 16:13:37,337][52866] Updated weights for policy 1, policy_version 35860 (0.0010) -[2023-10-15 16:13:37,543][52833] Updated weights for policy 0, policy_version 35760 (0.0008) -[2023-10-15 16:13:37,702][52866] Updated weights for policy 1, policy_version 35870 (0.0007) -[2023-10-15 16:13:37,909][52833] Updated weights for policy 0, policy_version 35770 (0.0007) -[2023-10-15 16:13:38,441][51532] Fps is (10 sec: 19660.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 73367552. Throughput: 0: 1792.3, 1: 1813.1. Samples: 18344232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:13:38,442][51532] Avg episode reward: [(0, '39.870'), (1, '39.800')] -[2023-10-15 16:13:41,459][52866] Updated weights for policy 1, policy_version 35880 (0.0009) -[2023-10-15 16:13:41,662][52833] Updated weights for policy 0, policy_version 35780 (0.0007) -[2023-10-15 16:13:41,817][52866] Updated weights for policy 1, policy_version 35890 (0.0009) -[2023-10-15 16:13:42,024][52833] Updated weights for policy 0, policy_version 35790 (0.0008) -[2023-10-15 16:13:42,190][52866] Updated weights for policy 1, policy_version 35900 (0.0008) -[2023-10-15 16:13:42,389][52833] Updated weights for policy 0, policy_version 35800 (0.0007) -[2023-10-15 16:13:43,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73433088. Throughput: 0: 1770.0, 1: 1796.9. Samples: 18364416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:13:43,442][51532] Avg episode reward: [(0, '40.110'), (1, '37.120')] -[2023-10-15 16:13:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000035904_36765696.pth... -[2023-10-15 16:13:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000035808_36667392.pth... -[2023-10-15 16:13:43,507][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000034144_34963456.pth -[2023-10-15 16:13:43,507][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000034208_35028992.pth -[2023-10-15 16:13:45,899][52866] Updated weights for policy 1, policy_version 35910 (0.0007) -[2023-10-15 16:13:46,101][52833] Updated weights for policy 0, policy_version 35810 (0.0007) -[2023-10-15 16:13:46,263][52866] Updated weights for policy 1, policy_version 35920 (0.0007) -[2023-10-15 16:13:46,470][52833] Updated weights for policy 0, policy_version 35820 (0.0007) -[2023-10-15 16:13:46,631][52866] Updated weights for policy 1, policy_version 35930 (0.0009) -[2023-10-15 16:13:46,847][52833] Updated weights for policy 0, policy_version 35830 (0.0008) -[2023-10-15 16:13:47,214][52833] Updated weights for policy 0, policy_version 35840 (0.0008) -[2023-10-15 16:13:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73498624. Throughput: 0: 1801.2, 1: 1813.7. Samples: 18376952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:13:48,441][51532] Avg episode reward: [(0, '39.430'), (1, '37.050')] -[2023-10-15 16:13:50,532][52866] Updated weights for policy 1, policy_version 35940 (0.0009) -[2023-10-15 16:13:50,905][52866] Updated weights for policy 1, policy_version 35950 (0.0008) -[2023-10-15 16:13:51,047][52833] Updated weights for policy 0, policy_version 35850 (0.0009) -[2023-10-15 16:13:51,271][52866] Updated weights for policy 1, policy_version 35960 (0.0008) -[2023-10-15 16:13:51,414][52833] Updated weights for policy 0, policy_version 35860 (0.0008) -[2023-10-15 16:13:51,795][52833] Updated weights for policy 0, policy_version 35870 (0.0007) -[2023-10-15 16:13:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 73564160. Throughput: 0: 1777.5, 1: 1791.4. Samples: 18396586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:13:53,442][51532] Avg episode reward: [(0, '38.990'), (1, '38.510')] -[2023-10-15 16:13:54,998][52866] Updated weights for policy 1, policy_version 35970 (0.0011) -[2023-10-15 16:13:55,364][52866] Updated weights for policy 1, policy_version 35980 (0.0008) -[2023-10-15 16:13:55,428][52833] Updated weights for policy 0, policy_version 35880 (0.0008) -[2023-10-15 16:13:55,733][52866] Updated weights for policy 1, policy_version 35990 (0.0009) -[2023-10-15 16:13:55,797][52833] Updated weights for policy 0, policy_version 35890 (0.0007) -[2023-10-15 16:13:56,091][52866] Updated weights for policy 1, policy_version 36000 (0.0009) -[2023-10-15 16:13:56,170][52833] Updated weights for policy 0, policy_version 35900 (0.0008) -[2023-10-15 16:13:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 73629696. Throughput: 0: 1781.2, 1: 1788.9. Samples: 18419064. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 16:13:58,442][51532] Avg episode reward: [(0, '41.970'), (1, '39.490')] -[2023-10-15 16:13:59,722][52866] Updated weights for policy 1, policy_version 36010 (0.0009) -[2023-10-15 16:14:00,021][52833] Updated weights for policy 0, policy_version 35910 (0.0009) -[2023-10-15 16:14:00,090][52866] Updated weights for policy 1, policy_version 36020 (0.0010) -[2023-10-15 16:14:00,385][52833] Updated weights for policy 0, policy_version 35920 (0.0007) -[2023-10-15 16:14:00,448][52866] Updated weights for policy 1, policy_version 36030 (0.0010) -[2023-10-15 16:14:00,756][52833] Updated weights for policy 0, policy_version 35930 (0.0008) -[2023-10-15 16:14:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 73695232. Throughput: 0: 1781.9, 1: 1788.9. Samples: 18428944. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 16:14:03,442][51532] Avg episode reward: [(0, '41.610'), (1, '39.620')] -[2023-10-15 16:14:04,235][52866] Updated weights for policy 1, policy_version 36040 (0.0008) -[2023-10-15 16:14:04,476][52833] Updated weights for policy 0, policy_version 35940 (0.0007) -[2023-10-15 16:14:04,608][52866] Updated weights for policy 1, policy_version 36050 (0.0007) -[2023-10-15 16:14:04,841][52833] Updated weights for policy 0, policy_version 35950 (0.0009) -[2023-10-15 16:14:04,969][52866] Updated weights for policy 1, policy_version 36060 (0.0008) -[2023-10-15 16:14:05,204][52833] Updated weights for policy 0, policy_version 35960 (0.0008) -[2023-10-15 16:14:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 73760768. Throughput: 0: 1781.6, 1: 1793.3. Samples: 18451156. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 16:14:08,442][51532] Avg episode reward: [(0, '42.480'), (1, '37.680')] -[2023-10-15 16:14:08,792][52866] Updated weights for policy 1, policy_version 36070 (0.0008) -[2023-10-15 16:14:08,853][52833] Updated weights for policy 0, policy_version 35970 (0.0008) -[2023-10-15 16:14:09,159][52866] Updated weights for policy 1, policy_version 36080 (0.0008) -[2023-10-15 16:14:09,224][52833] Updated weights for policy 0, policy_version 35980 (0.0009) -[2023-10-15 16:14:09,520][52866] Updated weights for policy 1, policy_version 36090 (0.0007) -[2023-10-15 16:14:09,589][52833] Updated weights for policy 0, policy_version 35990 (0.0007) -[2023-10-15 16:14:09,959][52833] Updated weights for policy 0, policy_version 36000 (0.0007) -[2023-10-15 16:14:13,203][52866] Updated weights for policy 1, policy_version 36100 (0.0007) -[2023-10-15 16:14:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 73826304. Throughput: 0: 1795.8, 1: 1802.7. Samples: 18473738. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 16:14:13,442][51532] Avg episode reward: [(0, '43.070'), (1, '38.590')] -[2023-10-15 16:14:13,573][52866] Updated weights for policy 1, policy_version 36110 (0.0007) -[2023-10-15 16:14:13,709][52833] Updated weights for policy 0, policy_version 36010 (0.0009) -[2023-10-15 16:14:13,945][52866] Updated weights for policy 1, policy_version 36120 (0.0009) -[2023-10-15 16:14:14,074][52833] Updated weights for policy 0, policy_version 36020 (0.0009) -[2023-10-15 16:14:14,458][52833] Updated weights for policy 0, policy_version 36030 (0.0009) -[2023-10-15 16:14:17,668][52866] Updated weights for policy 1, policy_version 36130 (0.0009) -[2023-10-15 16:14:18,050][52866] Updated weights for policy 1, policy_version 36140 (0.0008) -[2023-10-15 16:14:18,265][52833] Updated weights for policy 0, policy_version 36040 (0.0007) -[2023-10-15 16:14:18,404][52866] Updated weights for policy 1, policy_version 36150 (0.0007) -[2023-10-15 16:14:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 73891840. Throughput: 0: 1788.8, 1: 1786.6. Samples: 18483388. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 16:14:18,442][51532] Avg episode reward: [(0, '40.920'), (1, '40.330')] -[2023-10-15 16:14:18,636][52833] Updated weights for policy 0, policy_version 36050 (0.0009) -[2023-10-15 16:14:18,770][52866] Updated weights for policy 1, policy_version 36160 (0.0008) -[2023-10-15 16:14:18,995][52833] Updated weights for policy 0, policy_version 36060 (0.0008) -[2023-10-15 16:14:22,478][52866] Updated weights for policy 1, policy_version 36170 (0.0007) -[2023-10-15 16:14:22,781][52833] Updated weights for policy 0, policy_version 36070 (0.0008) -[2023-10-15 16:14:22,842][52866] Updated weights for policy 1, policy_version 36180 (0.0007) -[2023-10-15 16:14:23,151][52833] Updated weights for policy 0, policy_version 36080 (0.0007) -[2023-10-15 16:14:23,208][52866] Updated weights for policy 1, policy_version 36190 (0.0008) -[2023-10-15 16:14:23,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 73990144. Throughput: 0: 1790.9, 1: 1797.7. Samples: 18505716. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 16:14:23,441][51532] Avg episode reward: [(0, '42.710'), (1, '40.010')] -[2023-10-15 16:14:23,528][52833] Updated weights for policy 0, policy_version 36090 (0.0008) -[2023-10-15 16:14:27,019][52866] Updated weights for policy 1, policy_version 36200 (0.0007) -[2023-10-15 16:14:27,226][52833] Updated weights for policy 0, policy_version 36100 (0.0008) -[2023-10-15 16:14:27,380][52866] Updated weights for policy 1, policy_version 36210 (0.0008) -[2023-10-15 16:14:27,598][52833] Updated weights for policy 0, policy_version 36110 (0.0009) -[2023-10-15 16:14:27,741][52866] Updated weights for policy 1, policy_version 36220 (0.0008) -[2023-10-15 16:14:27,970][52833] Updated weights for policy 0, policy_version 36120 (0.0008) -[2023-10-15 16:14:28,441][51532] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 74088448. Throughput: 0: 1803.2, 1: 1783.6. Samples: 18525820. Policy #0 lag: (min: 17.0, avg: 17.7, max: 34.0) -[2023-10-15 16:14:28,442][51532] Avg episode reward: [(0, '43.490'), (1, '39.520')] -[2023-10-15 16:14:31,532][52866] Updated weights for policy 1, policy_version 36230 (0.0010) -[2023-10-15 16:14:31,626][52833] Updated weights for policy 0, policy_version 36130 (0.0008) -[2023-10-15 16:14:31,905][52866] Updated weights for policy 1, policy_version 36240 (0.0008) -[2023-10-15 16:14:31,999][52833] Updated weights for policy 0, policy_version 36140 (0.0007) -[2023-10-15 16:14:32,262][52866] Updated weights for policy 1, policy_version 36250 (0.0008) -[2023-10-15 16:14:32,356][52833] Updated weights for policy 0, policy_version 36150 (0.0008) -[2023-10-15 16:14:32,719][52833] Updated weights for policy 0, policy_version 36160 (0.0007) -[2023-10-15 16:14:33,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 74153984. Throughput: 0: 1787.0, 1: 1793.1. Samples: 18538058. Policy #0 lag: (min: 17.0, avg: 17.7, max: 34.0) -[2023-10-15 16:14:33,442][51532] Avg episode reward: [(0, '43.450'), (1, '42.230')] -[2023-10-15 16:14:35,989][52866] Updated weights for policy 1, policy_version 36260 (0.0008) -[2023-10-15 16:14:36,354][52866] Updated weights for policy 1, policy_version 36270 (0.0008) -[2023-10-15 16:14:36,598][52833] Updated weights for policy 0, policy_version 36170 (0.0008) -[2023-10-15 16:14:36,728][52866] Updated weights for policy 1, policy_version 36280 (0.0009) -[2023-10-15 16:14:36,973][52833] Updated weights for policy 0, policy_version 36180 (0.0009) -[2023-10-15 16:14:37,339][52833] Updated weights for policy 0, policy_version 36190 (0.0008) -[2023-10-15 16:14:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74219520. Throughput: 0: 1801.6, 1: 1792.2. Samples: 18558308. Policy #0 lag: (min: 17.0, avg: 17.7, max: 34.0) -[2023-10-15 16:14:38,442][51532] Avg episode reward: [(0, '41.370'), (1, '41.330')] -[2023-10-15 16:14:40,484][52866] Updated weights for policy 1, policy_version 36290 (0.0007) -[2023-10-15 16:14:40,847][52866] Updated weights for policy 1, policy_version 36300 (0.0007) -[2023-10-15 16:14:41,117][52833] Updated weights for policy 0, policy_version 36200 (0.0008) -[2023-10-15 16:14:41,217][52866] Updated weights for policy 1, policy_version 36310 (0.0008) -[2023-10-15 16:14:41,477][52833] Updated weights for policy 0, policy_version 36210 (0.0008) -[2023-10-15 16:14:41,581][52866] Updated weights for policy 1, policy_version 36320 (0.0009) -[2023-10-15 16:14:41,858][52833] Updated weights for policy 0, policy_version 36220 (0.0008) -[2023-10-15 16:14:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74285056. Throughput: 0: 1786.4, 1: 1788.5. Samples: 18579934. Policy #0 lag: (min: 17.0, avg: 17.7, max: 34.0) -[2023-10-15 16:14:43,442][51532] Avg episode reward: [(0, '39.560'), (1, '41.990')] -[2023-10-15 16:14:45,327][52866] Updated weights for policy 1, policy_version 36330 (0.0011) -[2023-10-15 16:14:45,693][52866] Updated weights for policy 1, policy_version 36340 (0.0008) -[2023-10-15 16:14:45,788][52833] Updated weights for policy 0, policy_version 36230 (0.0007) -[2023-10-15 16:14:46,057][52866] Updated weights for policy 1, policy_version 36350 (0.0008) -[2023-10-15 16:14:46,160][52833] Updated weights for policy 0, policy_version 36240 (0.0009) -[2023-10-15 16:14:46,528][52833] Updated weights for policy 0, policy_version 36250 (0.0008) -[2023-10-15 16:14:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74350592. Throughput: 0: 1807.4, 1: 1794.4. Samples: 18591026. Policy #0 lag: (min: 17.0, avg: 17.7, max: 34.0) -[2023-10-15 16:14:48,441][51532] Avg episode reward: [(0, '38.790'), (1, '44.190')] -[2023-10-15 16:14:49,908][52866] Updated weights for policy 1, policy_version 36360 (0.0010) -[2023-10-15 16:14:50,202][52833] Updated weights for policy 0, policy_version 36260 (0.0007) -[2023-10-15 16:14:50,278][52866] Updated weights for policy 1, policy_version 36370 (0.0008) -[2023-10-15 16:14:50,578][52833] Updated weights for policy 0, policy_version 36270 (0.0007) -[2023-10-15 16:14:50,636][52866] Updated weights for policy 1, policy_version 36380 (0.0008) -[2023-10-15 16:14:50,958][52833] Updated weights for policy 0, policy_version 36280 (0.0009) -[2023-10-15 16:14:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 74416128. Throughput: 0: 1787.0, 1: 1787.4. Samples: 18612004. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 16:14:53,441][51532] Avg episode reward: [(0, '39.940'), (1, '44.380')] -[2023-10-15 16:14:54,544][52866] Updated weights for policy 1, policy_version 36390 (0.0007) -[2023-10-15 16:14:54,732][52833] Updated weights for policy 0, policy_version 36290 (0.0007) -[2023-10-15 16:14:54,919][52866] Updated weights for policy 1, policy_version 36400 (0.0008) -[2023-10-15 16:14:55,091][52833] Updated weights for policy 0, policy_version 36300 (0.0008) -[2023-10-15 16:14:55,289][52866] Updated weights for policy 1, policy_version 36410 (0.0008) -[2023-10-15 16:14:55,463][52833] Updated weights for policy 0, policy_version 36310 (0.0008) -[2023-10-15 16:14:55,834][52833] Updated weights for policy 0, policy_version 36320 (0.0008) -[2023-10-15 16:14:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 74481664. Throughput: 0: 1782.2, 1: 1786.9. Samples: 18634344. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 16:14:58,441][51532] Avg episode reward: [(0, '38.850'), (1, '42.280')] -[2023-10-15 16:14:59,013][52866] Updated weights for policy 1, policy_version 36420 (0.0008) -[2023-10-15 16:14:59,377][52866] Updated weights for policy 1, policy_version 36430 (0.0009) -[2023-10-15 16:14:59,747][52866] Updated weights for policy 1, policy_version 36440 (0.0009) -[2023-10-15 16:14:59,761][52833] Updated weights for policy 0, policy_version 36330 (0.0009) -[2023-10-15 16:15:00,129][52833] Updated weights for policy 0, policy_version 36340 (0.0009) -[2023-10-15 16:15:00,497][52833] Updated weights for policy 0, policy_version 36350 (0.0007) -[2023-10-15 16:15:03,430][52866] Updated weights for policy 1, policy_version 36450 (0.0008) -[2023-10-15 16:15:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 74547200. Throughput: 0: 1786.3, 1: 1787.0. Samples: 18644186. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 16:15:03,441][51532] Avg episode reward: [(0, '39.800'), (1, '41.960')] -[2023-10-15 16:15:03,792][52866] Updated weights for policy 1, policy_version 36460 (0.0008) -[2023-10-15 16:15:04,097][52833] Updated weights for policy 0, policy_version 36360 (0.0007) -[2023-10-15 16:15:04,161][52866] Updated weights for policy 1, policy_version 36470 (0.0008) -[2023-10-15 16:15:04,465][52833] Updated weights for policy 0, policy_version 36370 (0.0008) -[2023-10-15 16:15:04,516][52866] Updated weights for policy 1, policy_version 36480 (0.0008) -[2023-10-15 16:15:04,830][52833] Updated weights for policy 0, policy_version 36380 (0.0008) -[2023-10-15 16:15:08,365][52866] Updated weights for policy 1, policy_version 36490 (0.0009) -[2023-10-15 16:15:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 74612736. Throughput: 0: 1793.1, 1: 1783.3. Samples: 18666654. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 16:15:08,441][51532] Avg episode reward: [(0, '39.220'), (1, '42.000')] -[2023-10-15 16:15:08,526][52833] Updated weights for policy 0, policy_version 36390 (0.0007) -[2023-10-15 16:15:08,727][52866] Updated weights for policy 1, policy_version 36500 (0.0007) -[2023-10-15 16:15:08,904][52833] Updated weights for policy 0, policy_version 36400 (0.0007) -[2023-10-15 16:15:09,100][52866] Updated weights for policy 1, policy_version 36510 (0.0007) -[2023-10-15 16:15:09,274][52833] Updated weights for policy 0, policy_version 36410 (0.0008) -[2023-10-15 16:15:12,943][52833] Updated weights for policy 0, policy_version 36420 (0.0008) -[2023-10-15 16:15:12,957][52866] Updated weights for policy 1, policy_version 36520 (0.0007) -[2023-10-15 16:15:13,298][52833] Updated weights for policy 0, policy_version 36430 (0.0007) -[2023-10-15 16:15:13,319][52866] Updated weights for policy 1, policy_version 36530 (0.0008) -[2023-10-15 16:15:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 74678272. Throughput: 0: 1812.4, 1: 1808.4. Samples: 18688756. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 16:15:13,442][51532] Avg episode reward: [(0, '39.930'), (1, '41.300')] -[2023-10-15 16:15:13,679][52833] Updated weights for policy 0, policy_version 36440 (0.0008) -[2023-10-15 16:15:13,690][52866] Updated weights for policy 1, policy_version 36540 (0.0008) -[2023-10-15 16:15:17,421][52866] Updated weights for policy 1, policy_version 36550 (0.0007) -[2023-10-15 16:15:17,449][52833] Updated weights for policy 0, policy_version 36450 (0.0008) -[2023-10-15 16:15:17,786][52866] Updated weights for policy 1, policy_version 36560 (0.0010) -[2023-10-15 16:15:17,811][52833] Updated weights for policy 0, policy_version 36460 (0.0008) -[2023-10-15 16:15:18,163][52866] Updated weights for policy 1, policy_version 36570 (0.0008) -[2023-10-15 16:15:18,180][52833] Updated weights for policy 0, policy_version 36470 (0.0007) -[2023-10-15 16:15:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 74776576. Throughput: 0: 1794.9, 1: 1782.5. Samples: 18699044. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 16:15:18,441][51532] Avg episode reward: [(0, '38.740'), (1, '40.040')] -[2023-10-15 16:15:18,551][52833] Updated weights for policy 0, policy_version 36480 (0.0008) -[2023-10-15 16:15:22,052][52866] Updated weights for policy 1, policy_version 36580 (0.0008) -[2023-10-15 16:15:22,333][52833] Updated weights for policy 0, policy_version 36490 (0.0009) -[2023-10-15 16:15:22,426][52866] Updated weights for policy 1, policy_version 36590 (0.0008) -[2023-10-15 16:15:22,686][52833] Updated weights for policy 0, policy_version 36500 (0.0009) -[2023-10-15 16:15:22,784][52866] Updated weights for policy 1, policy_version 36600 (0.0009) -[2023-10-15 16:15:23,053][52833] Updated weights for policy 0, policy_version 36510 (0.0007) -[2023-10-15 16:15:23,441][51532] Fps is (10 sec: 19661.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 74874880. Throughput: 0: 1813.3, 1: 1803.5. Samples: 18721064. Policy #0 lag: (min: 8.0, avg: 27.4, max: 40.0) -[2023-10-15 16:15:23,442][51532] Avg episode reward: [(0, '39.460'), (1, '38.370')] -[2023-10-15 16:15:26,572][52866] Updated weights for policy 1, policy_version 36610 (0.0007) -[2023-10-15 16:15:26,648][52833] Updated weights for policy 0, policy_version 36520 (0.0007) -[2023-10-15 16:15:26,927][52866] Updated weights for policy 1, policy_version 36620 (0.0007) -[2023-10-15 16:15:27,021][52833] Updated weights for policy 0, policy_version 36530 (0.0009) -[2023-10-15 16:15:27,297][52866] Updated weights for policy 1, policy_version 36630 (0.0007) -[2023-10-15 16:15:27,382][52833] Updated weights for policy 0, policy_version 36540 (0.0009) -[2023-10-15 16:15:27,662][52866] Updated weights for policy 1, policy_version 36640 (0.0007) -[2023-10-15 16:15:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 74940416. Throughput: 0: 1801.9, 1: 1770.1. Samples: 18740672. Policy #0 lag: (min: 8.0, avg: 27.4, max: 40.0) -[2023-10-15 16:15:28,441][51532] Avg episode reward: [(0, '39.290'), (1, '36.400')] -[2023-10-15 16:15:31,172][52833] Updated weights for policy 0, policy_version 36550 (0.0009) -[2023-10-15 16:15:31,346][52866] Updated weights for policy 1, policy_version 36650 (0.0008) -[2023-10-15 16:15:31,534][52833] Updated weights for policy 0, policy_version 36560 (0.0008) -[2023-10-15 16:15:31,707][52866] Updated weights for policy 1, policy_version 36660 (0.0008) -[2023-10-15 16:15:31,905][52833] Updated weights for policy 0, policy_version 36570 (0.0008) -[2023-10-15 16:15:32,079][52866] Updated weights for policy 1, policy_version 36670 (0.0009) -[2023-10-15 16:15:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75005952. Throughput: 0: 1811.1, 1: 1796.3. Samples: 18753356. Policy #0 lag: (min: 8.0, avg: 27.4, max: 40.0) -[2023-10-15 16:15:33,442][51532] Avg episode reward: [(0, '39.120'), (1, '37.240')] -[2023-10-15 16:15:35,588][52833] Updated weights for policy 0, policy_version 36580 (0.0007) -[2023-10-15 16:15:35,959][52833] Updated weights for policy 0, policy_version 36590 (0.0008) -[2023-10-15 16:15:35,976][52866] Updated weights for policy 1, policy_version 36680 (0.0009) -[2023-10-15 16:15:36,330][52833] Updated weights for policy 0, policy_version 36600 (0.0008) -[2023-10-15 16:15:36,338][52866] Updated weights for policy 1, policy_version 36690 (0.0008) -[2023-10-15 16:15:36,715][52866] Updated weights for policy 1, policy_version 36700 (0.0008) -[2023-10-15 16:15:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75071488. Throughput: 0: 1797.4, 1: 1767.6. Samples: 18772430. Policy #0 lag: (min: 8.0, avg: 27.4, max: 40.0) -[2023-10-15 16:15:38,442][51532] Avg episode reward: [(0, '39.750'), (1, '39.230')] -[2023-10-15 16:15:39,961][52833] Updated weights for policy 0, policy_version 36610 (0.0008) -[2023-10-15 16:15:40,333][52833] Updated weights for policy 0, policy_version 36620 (0.0007) -[2023-10-15 16:15:40,465][52866] Updated weights for policy 1, policy_version 36710 (0.0008) -[2023-10-15 16:15:40,697][52833] Updated weights for policy 0, policy_version 36630 (0.0007) -[2023-10-15 16:15:40,834][52866] Updated weights for policy 1, policy_version 36720 (0.0008) -[2023-10-15 16:15:41,058][52833] Updated weights for policy 0, policy_version 36640 (0.0007) -[2023-10-15 16:15:41,203][52866] Updated weights for policy 1, policy_version 36730 (0.0009) -[2023-10-15 16:15:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 75137024. Throughput: 0: 1800.2, 1: 1768.7. Samples: 18794942. Policy #0 lag: (min: 8.0, avg: 27.4, max: 40.0) -[2023-10-15 16:15:43,442][51532] Avg episode reward: [(0, '40.930'), (1, '38.940')] -[2023-10-15 16:15:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000036640_37519360.pth... -[2023-10-15 16:15:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000036736_37617664.pth... -[2023-10-15 16:15:43,481][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000034976_35815424.pth -[2023-10-15 16:15:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000035072_35913728.pth -[2023-10-15 16:15:44,894][52866] Updated weights for policy 1, policy_version 36740 (0.0010) -[2023-10-15 16:15:45,011][52833] Updated weights for policy 0, policy_version 36650 (0.0010) -[2023-10-15 16:15:45,266][52866] Updated weights for policy 1, policy_version 36750 (0.0008) -[2023-10-15 16:15:45,376][52833] Updated weights for policy 0, policy_version 36660 (0.0008) -[2023-10-15 16:15:45,632][52866] Updated weights for policy 1, policy_version 36760 (0.0007) -[2023-10-15 16:15:45,747][52833] Updated weights for policy 0, policy_version 36670 (0.0008) -[2023-10-15 16:15:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 75202560. Throughput: 0: 1798.0, 1: 1771.3. Samples: 18804804. Policy #0 lag: (min: 1.0, avg: 1.2, max: 10.0) -[2023-10-15 16:15:48,441][51532] Avg episode reward: [(0, '41.910'), (1, '35.860')] -[2023-10-15 16:15:49,530][52866] Updated weights for policy 1, policy_version 36770 (0.0007) -[2023-10-15 16:15:49,616][52833] Updated weights for policy 0, policy_version 36680 (0.0010) -[2023-10-15 16:15:49,898][52866] Updated weights for policy 1, policy_version 36780 (0.0009) -[2023-10-15 16:15:49,981][52833] Updated weights for policy 0, policy_version 36690 (0.0009) -[2023-10-15 16:15:50,259][52866] Updated weights for policy 1, policy_version 36790 (0.0009) -[2023-10-15 16:15:50,351][52833] Updated weights for policy 0, policy_version 36700 (0.0009) -[2023-10-15 16:15:50,624][52866] Updated weights for policy 1, policy_version 36800 (0.0008) -[2023-10-15 16:15:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 75268096. Throughput: 0: 1787.0, 1: 1770.5. Samples: 18826742. Policy #0 lag: (min: 1.0, avg: 1.2, max: 10.0) -[2023-10-15 16:15:53,442][51532] Avg episode reward: [(0, '42.350'), (1, '37.470')] -[2023-10-15 16:15:54,182][52833] Updated weights for policy 0, policy_version 36710 (0.0008) -[2023-10-15 16:15:54,435][52866] Updated weights for policy 1, policy_version 36810 (0.0010) -[2023-10-15 16:15:54,547][52833] Updated weights for policy 0, policy_version 36720 (0.0008) -[2023-10-15 16:15:54,796][52866] Updated weights for policy 1, policy_version 36820 (0.0008) -[2023-10-15 16:15:54,916][52833] Updated weights for policy 0, policy_version 36730 (0.0008) -[2023-10-15 16:15:55,157][52866] Updated weights for policy 1, policy_version 36830 (0.0009) -[2023-10-15 16:15:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 75333632. Throughput: 0: 1789.4, 1: 1776.5. Samples: 18849224. Policy #0 lag: (min: 1.0, avg: 1.2, max: 10.0) -[2023-10-15 16:15:58,441][51532] Avg episode reward: [(0, '43.280'), (1, '37.370')] -[2023-10-15 16:15:58,605][52833] Updated weights for policy 0, policy_version 36740 (0.0010) -[2023-10-15 16:15:58,976][52866] Updated weights for policy 1, policy_version 36840 (0.0007) -[2023-10-15 16:15:58,977][52833] Updated weights for policy 0, policy_version 36750 (0.0008) -[2023-10-15 16:15:59,341][52833] Updated weights for policy 0, policy_version 36760 (0.0008) -[2023-10-15 16:15:59,351][52866] Updated weights for policy 1, policy_version 36850 (0.0008) -[2023-10-15 16:15:59,720][52866] Updated weights for policy 1, policy_version 36860 (0.0009) -[2023-10-15 16:16:02,953][52833] Updated weights for policy 0, policy_version 36770 (0.0008) -[2023-10-15 16:16:03,333][52833] Updated weights for policy 0, policy_version 36780 (0.0008) -[2023-10-15 16:16:03,439][52866] Updated weights for policy 1, policy_version 36870 (0.0007) -[2023-10-15 16:16:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 75399168. Throughput: 0: 1790.2, 1: 1765.2. Samples: 18859034. Policy #0 lag: (min: 1.0, avg: 1.2, max: 10.0) -[2023-10-15 16:16:03,441][51532] Avg episode reward: [(0, '43.700'), (1, '35.960')] -[2023-10-15 16:16:03,705][52833] Updated weights for policy 0, policy_version 36790 (0.0008) -[2023-10-15 16:16:03,809][52866] Updated weights for policy 1, policy_version 36880 (0.0007) -[2023-10-15 16:16:04,068][52833] Updated weights for policy 0, policy_version 36800 (0.0010) -[2023-10-15 16:16:04,175][52866] Updated weights for policy 1, policy_version 36890 (0.0009) -[2023-10-15 16:16:07,876][52866] Updated weights for policy 1, policy_version 36900 (0.0009) -[2023-10-15 16:16:07,911][52833] Updated weights for policy 0, policy_version 36810 (0.0008) -[2023-10-15 16:16:08,244][52866] Updated weights for policy 1, policy_version 36910 (0.0007) -[2023-10-15 16:16:08,284][52833] Updated weights for policy 0, policy_version 36820 (0.0008) -[2023-10-15 16:16:08,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 75464704. Throughput: 0: 1788.6, 1: 1771.1. Samples: 18881254. Policy #0 lag: (min: 1.0, avg: 1.2, max: 10.0) -[2023-10-15 16:16:08,442][51532] Avg episode reward: [(0, '42.550'), (1, '34.810')] -[2023-10-15 16:16:08,622][52866] Updated weights for policy 1, policy_version 36920 (0.0009) -[2023-10-15 16:16:08,650][52833] Updated weights for policy 0, policy_version 36830 (0.0009) -[2023-10-15 16:16:12,297][52833] Updated weights for policy 0, policy_version 36840 (0.0008) -[2023-10-15 16:16:12,422][52866] Updated weights for policy 1, policy_version 36930 (0.0007) -[2023-10-15 16:16:12,670][52833] Updated weights for policy 0, policy_version 36850 (0.0008) -[2023-10-15 16:16:12,784][52866] Updated weights for policy 1, policy_version 36940 (0.0008) -[2023-10-15 16:16:13,040][52833] Updated weights for policy 0, policy_version 36860 (0.0008) -[2023-10-15 16:16:13,147][52866] Updated weights for policy 1, policy_version 36950 (0.0007) -[2023-10-15 16:16:13,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 75563008. Throughput: 0: 1793.9, 1: 1790.7. Samples: 18901980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:16:13,442][51532] Avg episode reward: [(0, '42.580'), (1, '33.390')] -[2023-10-15 16:16:13,516][52866] Updated weights for policy 1, policy_version 36960 (0.0009) -[2023-10-15 16:16:16,795][52833] Updated weights for policy 0, policy_version 36870 (0.0010) -[2023-10-15 16:16:17,163][52833] Updated weights for policy 0, policy_version 36880 (0.0008) -[2023-10-15 16:16:17,349][52866] Updated weights for policy 1, policy_version 36970 (0.0007) -[2023-10-15 16:16:17,536][52833] Updated weights for policy 0, policy_version 36890 (0.0009) -[2023-10-15 16:16:17,711][52866] Updated weights for policy 1, policy_version 36980 (0.0007) -[2023-10-15 16:16:18,075][52866] Updated weights for policy 1, policy_version 36990 (0.0008) -[2023-10-15 16:16:18,441][51532] Fps is (10 sec: 19661.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 75661312. Throughput: 0: 1784.1, 1: 1773.9. Samples: 18913470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:16:18,442][51532] Avg episode reward: [(0, '44.980'), (1, '32.760')] -[2023-10-15 16:16:18,443][52410] Saving new best policy, reward=44.980! -[2023-10-15 16:16:21,325][52833] Updated weights for policy 0, policy_version 36900 (0.0008) -[2023-10-15 16:16:21,684][52833] Updated weights for policy 0, policy_version 36910 (0.0011) -[2023-10-15 16:16:21,849][52866] Updated weights for policy 1, policy_version 37000 (0.0007) -[2023-10-15 16:16:22,058][52833] Updated weights for policy 0, policy_version 36920 (0.0008) -[2023-10-15 16:16:22,213][52866] Updated weights for policy 1, policy_version 37010 (0.0007) -[2023-10-15 16:16:22,586][52866] Updated weights for policy 1, policy_version 37020 (0.0008) -[2023-10-15 16:16:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 75726848. Throughput: 0: 1805.3, 1: 1800.1. Samples: 18934672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:16:23,442][51532] Avg episode reward: [(0, '44.640'), (1, '34.130')] -[2023-10-15 16:16:25,779][52833] Updated weights for policy 0, policy_version 36930 (0.0009) -[2023-10-15 16:16:26,150][52833] Updated weights for policy 0, policy_version 36940 (0.0008) -[2023-10-15 16:16:26,518][52833] Updated weights for policy 0, policy_version 36950 (0.0009) -[2023-10-15 16:16:26,591][52866] Updated weights for policy 1, policy_version 37030 (0.0010) -[2023-10-15 16:16:26,890][52833] Updated weights for policy 0, policy_version 36960 (0.0009) -[2023-10-15 16:16:26,970][52866] Updated weights for policy 1, policy_version 37040 (0.0008) -[2023-10-15 16:16:27,348][52866] Updated weights for policy 1, policy_version 37050 (0.0008) -[2023-10-15 16:16:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 75792384. Throughput: 0: 1787.0, 1: 1773.8. Samples: 18955178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:16:28,442][51532] Avg episode reward: [(0, '42.830'), (1, '32.120')] -[2023-10-15 16:16:30,867][52866] Updated weights for policy 1, policy_version 37060 (0.0007) -[2023-10-15 16:16:30,872][52833] Updated weights for policy 0, policy_version 36970 (0.0007) -[2023-10-15 16:16:31,237][52866] Updated weights for policy 1, policy_version 37070 (0.0008) -[2023-10-15 16:16:31,242][52833] Updated weights for policy 0, policy_version 36980 (0.0008) -[2023-10-15 16:16:31,599][52866] Updated weights for policy 1, policy_version 37080 (0.0009) -[2023-10-15 16:16:31,605][52833] Updated weights for policy 0, policy_version 36990 (0.0008) -[2023-10-15 16:16:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 75857920. Throughput: 0: 1806.2, 1: 1803.1. Samples: 18967220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:16:33,442][51532] Avg episode reward: [(0, '42.390'), (1, '33.920')] -[2023-10-15 16:16:35,155][52833] Updated weights for policy 0, policy_version 37000 (0.0008) -[2023-10-15 16:16:35,373][52866] Updated weights for policy 1, policy_version 37090 (0.0008) -[2023-10-15 16:16:35,530][52833] Updated weights for policy 0, policy_version 37010 (0.0008) -[2023-10-15 16:16:35,736][52866] Updated weights for policy 1, policy_version 37100 (0.0008) -[2023-10-15 16:16:35,890][52833] Updated weights for policy 0, policy_version 37020 (0.0008) -[2023-10-15 16:16:36,106][52866] Updated weights for policy 1, policy_version 37110 (0.0009) -[2023-10-15 16:16:36,474][52866] Updated weights for policy 1, policy_version 37120 (0.0009) -[2023-10-15 16:16:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 75923456. Throughput: 0: 1792.1, 1: 1776.8. Samples: 18987340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:16:38,441][51532] Avg episode reward: [(0, '43.620'), (1, '35.100')] -[2023-10-15 16:16:39,755][52833] Updated weights for policy 0, policy_version 37030 (0.0010) -[2023-10-15 16:16:40,121][52833] Updated weights for policy 0, policy_version 37040 (0.0009) -[2023-10-15 16:16:40,245][52866] Updated weights for policy 1, policy_version 37130 (0.0009) -[2023-10-15 16:16:40,497][52833] Updated weights for policy 0, policy_version 37050 (0.0009) -[2023-10-15 16:16:40,608][52866] Updated weights for policy 1, policy_version 37140 (0.0009) -[2023-10-15 16:16:40,974][52866] Updated weights for policy 1, policy_version 37150 (0.0010) -[2023-10-15 16:16:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 75988992. Throughput: 0: 1789.7, 1: 1783.0. Samples: 19009996. Policy #0 lag: (min: 28.0, avg: 52.4, max: 56.0) -[2023-10-15 16:16:43,441][51532] Avg episode reward: [(0, '41.440'), (1, '36.540')] -[2023-10-15 16:16:44,178][52833] Updated weights for policy 0, policy_version 37060 (0.0008) -[2023-10-15 16:16:44,546][52833] Updated weights for policy 0, policy_version 37070 (0.0008) -[2023-10-15 16:16:44,651][52866] Updated weights for policy 1, policy_version 37160 (0.0007) -[2023-10-15 16:16:44,909][52833] Updated weights for policy 0, policy_version 37080 (0.0007) -[2023-10-15 16:16:45,010][52866] Updated weights for policy 1, policy_version 37170 (0.0009) -[2023-10-15 16:16:45,380][52866] Updated weights for policy 1, policy_version 37180 (0.0008) -[2023-10-15 16:16:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 76054528. Throughput: 0: 1786.8, 1: 1786.5. Samples: 19019830. Policy #0 lag: (min: 28.0, avg: 52.4, max: 56.0) -[2023-10-15 16:16:48,441][51532] Avg episode reward: [(0, '43.670'), (1, '37.700')] -[2023-10-15 16:16:48,748][52833] Updated weights for policy 0, policy_version 37090 (0.0008) -[2023-10-15 16:16:49,121][52833] Updated weights for policy 0, policy_version 37100 (0.0007) -[2023-10-15 16:16:49,348][52866] Updated weights for policy 1, policy_version 37190 (0.0007) -[2023-10-15 16:16:49,490][52833] Updated weights for policy 0, policy_version 37110 (0.0007) -[2023-10-15 16:16:49,717][52866] Updated weights for policy 1, policy_version 37200 (0.0007) -[2023-10-15 16:16:49,856][52833] Updated weights for policy 0, policy_version 37120 (0.0007) -[2023-10-15 16:16:50,075][52866] Updated weights for policy 1, policy_version 37210 (0.0008) -[2023-10-15 16:16:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 76120064. Throughput: 0: 1787.7, 1: 1785.7. Samples: 19042058. Policy #0 lag: (min: 28.0, avg: 52.4, max: 56.0) -[2023-10-15 16:16:53,441][51532] Avg episode reward: [(0, '45.740'), (1, '38.770')] -[2023-10-15 16:16:53,574][52833] Updated weights for policy 0, policy_version 37130 (0.0008) -[2023-10-15 16:16:53,713][52866] Updated weights for policy 1, policy_version 37220 (0.0007) -[2023-10-15 16:16:53,930][52833] Updated weights for policy 0, policy_version 37140 (0.0009) -[2023-10-15 16:16:54,085][52866] Updated weights for policy 1, policy_version 37230 (0.0008) -[2023-10-15 16:16:54,297][52833] Updated weights for policy 0, policy_version 37150 (0.0008) -[2023-10-15 16:16:54,369][52410] Saving new best policy, reward=45.740! -[2023-10-15 16:16:54,453][52866] Updated weights for policy 1, policy_version 37240 (0.0008) -[2023-10-15 16:16:58,078][52833] Updated weights for policy 0, policy_version 37160 (0.0007) -[2023-10-15 16:16:58,198][52866] Updated weights for policy 1, policy_version 37250 (0.0010) -[2023-10-15 16:16:58,437][52833] Updated weights for policy 0, policy_version 37170 (0.0008) -[2023-10-15 16:16:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 76185600. Throughput: 0: 1809.5, 1: 1808.7. Samples: 19064798. Policy #0 lag: (min: 28.0, avg: 52.4, max: 56.0) -[2023-10-15 16:16:58,441][51532] Avg episode reward: [(0, '40.820'), (1, '39.110')] -[2023-10-15 16:16:58,561][52866] Updated weights for policy 1, policy_version 37260 (0.0010) -[2023-10-15 16:16:58,800][52833] Updated weights for policy 0, policy_version 37180 (0.0007) -[2023-10-15 16:16:58,923][52866] Updated weights for policy 1, policy_version 37270 (0.0008) -[2023-10-15 16:16:59,292][52866] Updated weights for policy 1, policy_version 37280 (0.0010) -[2023-10-15 16:17:02,456][52833] Updated weights for policy 0, policy_version 37190 (0.0008) -[2023-10-15 16:17:02,824][52833] Updated weights for policy 0, policy_version 37200 (0.0008) -[2023-10-15 16:17:02,977][52866] Updated weights for policy 1, policy_version 37290 (0.0008) -[2023-10-15 16:17:03,183][52833] Updated weights for policy 0, policy_version 37210 (0.0008) -[2023-10-15 16:17:03,335][52866] Updated weights for policy 1, policy_version 37300 (0.0009) -[2023-10-15 16:17:03,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 76283904. Throughput: 0: 1794.2, 1: 1794.2. Samples: 19074950. Policy #0 lag: (min: 28.0, avg: 52.4, max: 56.0) -[2023-10-15 16:17:03,442][51532] Avg episode reward: [(0, '40.600'), (1, '39.060')] -[2023-10-15 16:17:03,711][52866] Updated weights for policy 1, policy_version 37310 (0.0008) -[2023-10-15 16:17:06,866][52833] Updated weights for policy 0, policy_version 37220 (0.0010) -[2023-10-15 16:17:07,243][52833] Updated weights for policy 0, policy_version 37230 (0.0011) -[2023-10-15 16:17:07,601][52866] Updated weights for policy 1, policy_version 37320 (0.0009) -[2023-10-15 16:17:07,613][52833] Updated weights for policy 0, policy_version 37240 (0.0008) -[2023-10-15 16:17:07,967][52866] Updated weights for policy 1, policy_version 37330 (0.0008) -[2023-10-15 16:17:08,338][52866] Updated weights for policy 1, policy_version 37340 (0.0008) -[2023-10-15 16:17:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 76349440. Throughput: 0: 1810.2, 1: 1801.2. Samples: 19097184. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) -[2023-10-15 16:17:08,441][51532] Avg episode reward: [(0, '42.440'), (1, '39.860')] -[2023-10-15 16:17:11,312][52833] Updated weights for policy 0, policy_version 37250 (0.0008) -[2023-10-15 16:17:11,681][52833] Updated weights for policy 0, policy_version 37260 (0.0007) -[2023-10-15 16:17:12,055][52833] Updated weights for policy 0, policy_version 37270 (0.0007) -[2023-10-15 16:17:12,138][52866] Updated weights for policy 1, policy_version 37350 (0.0009) -[2023-10-15 16:17:12,421][52833] Updated weights for policy 0, policy_version 37280 (0.0007) -[2023-10-15 16:17:12,505][52866] Updated weights for policy 1, policy_version 37360 (0.0007) -[2023-10-15 16:17:12,883][52866] Updated weights for policy 1, policy_version 37370 (0.0008) -[2023-10-15 16:17:13,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 76447744. Throughput: 0: 1797.5, 1: 1802.8. Samples: 19117188. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) -[2023-10-15 16:17:13,442][51532] Avg episode reward: [(0, '43.820'), (1, '40.040')] -[2023-10-15 16:17:16,277][52833] Updated weights for policy 0, policy_version 37290 (0.0009) -[2023-10-15 16:17:16,482][52866] Updated weights for policy 1, policy_version 37380 (0.0008) -[2023-10-15 16:17:16,644][52833] Updated weights for policy 0, policy_version 37300 (0.0008) -[2023-10-15 16:17:16,852][52866] Updated weights for policy 1, policy_version 37390 (0.0008) -[2023-10-15 16:17:17,010][52833] Updated weights for policy 0, policy_version 37310 (0.0010) -[2023-10-15 16:17:17,217][52866] Updated weights for policy 1, policy_version 37400 (0.0007) -[2023-10-15 16:17:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 76513280. Throughput: 0: 1807.1, 1: 1802.0. Samples: 19129628. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) -[2023-10-15 16:17:18,442][51532] Avg episode reward: [(0, '44.300'), (1, '38.460')] -[2023-10-15 16:17:20,906][52833] Updated weights for policy 0, policy_version 37320 (0.0007) -[2023-10-15 16:17:20,928][52866] Updated weights for policy 1, policy_version 37410 (0.0007) -[2023-10-15 16:17:21,268][52833] Updated weights for policy 0, policy_version 37330 (0.0009) -[2023-10-15 16:17:21,304][52866] Updated weights for policy 1, policy_version 37420 (0.0008) -[2023-10-15 16:17:21,632][52833] Updated weights for policy 0, policy_version 37340 (0.0009) -[2023-10-15 16:17:21,667][52866] Updated weights for policy 1, policy_version 37430 (0.0007) -[2023-10-15 16:17:22,035][52866] Updated weights for policy 1, policy_version 37440 (0.0008) -[2023-10-15 16:17:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 76578816. Throughput: 0: 1792.8, 1: 1801.7. Samples: 19149090. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) -[2023-10-15 16:17:23,442][51532] Avg episode reward: [(0, '42.910'), (1, '36.860')] -[2023-10-15 16:17:25,417][52833] Updated weights for policy 0, policy_version 37350 (0.0008) -[2023-10-15 16:17:25,789][52833] Updated weights for policy 0, policy_version 37360 (0.0008) -[2023-10-15 16:17:25,827][52866] Updated weights for policy 1, policy_version 37450 (0.0007) -[2023-10-15 16:17:26,147][52833] Updated weights for policy 0, policy_version 37370 (0.0009) -[2023-10-15 16:17:26,185][52866] Updated weights for policy 1, policy_version 37460 (0.0009) -[2023-10-15 16:17:26,550][52866] Updated weights for policy 1, policy_version 37470 (0.0008) -[2023-10-15 16:17:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 76644352. Throughput: 0: 1790.3, 1: 1796.0. Samples: 19171378. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) -[2023-10-15 16:17:28,442][51532] Avg episode reward: [(0, '44.850'), (1, '38.210')] -[2023-10-15 16:17:29,807][52833] Updated weights for policy 0, policy_version 37380 (0.0009) -[2023-10-15 16:17:30,169][52833] Updated weights for policy 0, policy_version 37390 (0.0007) -[2023-10-15 16:17:30,281][52866] Updated weights for policy 1, policy_version 37480 (0.0008) -[2023-10-15 16:17:30,539][52833] Updated weights for policy 0, policy_version 37400 (0.0008) -[2023-10-15 16:17:30,643][52866] Updated weights for policy 1, policy_version 37490 (0.0009) -[2023-10-15 16:17:31,015][52866] Updated weights for policy 1, policy_version 37500 (0.0008) -[2023-10-15 16:17:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 76709888. Throughput: 0: 1792.5, 1: 1801.0. Samples: 19181536. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) -[2023-10-15 16:17:33,442][51532] Avg episode reward: [(0, '40.750'), (1, '38.470')] -[2023-10-15 16:17:34,317][52833] Updated weights for policy 0, policy_version 37410 (0.0008) -[2023-10-15 16:17:34,695][52833] Updated weights for policy 0, policy_version 37420 (0.0010) -[2023-10-15 16:17:34,740][52866] Updated weights for policy 1, policy_version 37510 (0.0010) -[2023-10-15 16:17:35,054][52833] Updated weights for policy 0, policy_version 37430 (0.0007) -[2023-10-15 16:17:35,115][52866] Updated weights for policy 1, policy_version 37520 (0.0009) -[2023-10-15 16:17:35,421][52833] Updated weights for policy 0, policy_version 37440 (0.0008) -[2023-10-15 16:17:35,473][52866] Updated weights for policy 1, policy_version 37530 (0.0008) -[2023-10-15 16:17:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 76775424. Throughput: 0: 1791.2, 1: 1793.0. Samples: 19203348. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 16:17:38,442][51532] Avg episode reward: [(0, '41.650'), (1, '41.350')] -[2023-10-15 16:17:39,225][52833] Updated weights for policy 0, policy_version 37450 (0.0007) -[2023-10-15 16:17:39,273][52866] Updated weights for policy 1, policy_version 37540 (0.0010) -[2023-10-15 16:17:39,589][52833] Updated weights for policy 0, policy_version 37460 (0.0007) -[2023-10-15 16:17:39,634][52866] Updated weights for policy 1, policy_version 37550 (0.0008) -[2023-10-15 16:17:39,959][52833] Updated weights for policy 0, policy_version 37470 (0.0007) -[2023-10-15 16:17:40,000][52866] Updated weights for policy 1, policy_version 37560 (0.0007) -[2023-10-15 16:17:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 76840960. Throughput: 0: 1797.1, 1: 1785.5. Samples: 19226020. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 16:17:43,442][51532] Avg episode reward: [(0, '41.820'), (1, '41.650')] -[2023-10-15 16:17:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000037568_38469632.pth... -[2023-10-15 16:17:43,492][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000035904_36765696.pth -[2023-10-15 16:17:43,630][52833] Updated weights for policy 0, policy_version 37480 (0.0007) -[2023-10-15 16:17:43,866][52866] Updated weights for policy 1, policy_version 37570 (0.0007) -[2023-10-15 16:17:44,001][52833] Updated weights for policy 0, policy_version 37490 (0.0007) -[2023-10-15 16:17:44,241][52866] Updated weights for policy 1, policy_version 37580 (0.0008) -[2023-10-15 16:17:44,368][52833] Updated weights for policy 0, policy_version 37500 (0.0009) -[2023-10-15 16:17:44,510][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000037504_38404096.pth... -[2023-10-15 16:17:44,539][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000035808_36667392.pth -[2023-10-15 16:17:44,613][52866] Updated weights for policy 1, policy_version 37590 (0.0009) -[2023-10-15 16:17:44,972][52866] Updated weights for policy 1, policy_version 37600 (0.0009) -[2023-10-15 16:17:48,065][52833] Updated weights for policy 0, policy_version 37510 (0.0008) -[2023-10-15 16:17:48,436][52833] Updated weights for policy 0, policy_version 37520 (0.0008) -[2023-10-15 16:17:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 76906496. Throughput: 0: 1790.7, 1: 1784.0. Samples: 19235812. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 16:17:48,441][51532] Avg episode reward: [(0, '39.990'), (1, '42.770')] -[2023-10-15 16:17:48,780][52866] Updated weights for policy 1, policy_version 37610 (0.0008) -[2023-10-15 16:17:48,794][52833] Updated weights for policy 0, policy_version 37530 (0.0010) -[2023-10-15 16:17:49,147][52866] Updated weights for policy 1, policy_version 37620 (0.0007) -[2023-10-15 16:17:49,517][52866] Updated weights for policy 1, policy_version 37630 (0.0008) -[2023-10-15 16:17:52,419][52833] Updated weights for policy 0, policy_version 37540 (0.0009) -[2023-10-15 16:17:52,789][52833] Updated weights for policy 0, policy_version 37550 (0.0011) -[2023-10-15 16:17:53,158][52833] Updated weights for policy 0, policy_version 37560 (0.0009) -[2023-10-15 16:17:53,269][52866] Updated weights for policy 1, policy_version 37640 (0.0008) -[2023-10-15 16:17:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 76972032. Throughput: 0: 1794.9, 1: 1785.6. Samples: 19258310. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 16:17:53,441][51532] Avg episode reward: [(0, '39.580'), (1, '42.280')] -[2023-10-15 16:17:53,628][52866] Updated weights for policy 1, policy_version 37650 (0.0008) -[2023-10-15 16:17:54,010][52866] Updated weights for policy 1, policy_version 37660 (0.0009) -[2023-10-15 16:17:56,970][52833] Updated weights for policy 0, policy_version 37570 (0.0008) -[2023-10-15 16:17:57,353][52833] Updated weights for policy 0, policy_version 37580 (0.0010) -[2023-10-15 16:17:57,715][52833] Updated weights for policy 0, policy_version 37590 (0.0008) -[2023-10-15 16:17:57,876][52866] Updated weights for policy 1, policy_version 37670 (0.0008) -[2023-10-15 16:17:58,076][52833] Updated weights for policy 0, policy_version 37600 (0.0009) -[2023-10-15 16:17:58,255][52866] Updated weights for policy 1, policy_version 37680 (0.0007) -[2023-10-15 16:17:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 77070336. Throughput: 0: 1796.8, 1: 1803.0. Samples: 19279180. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) -[2023-10-15 16:17:58,441][51532] Avg episode reward: [(0, '40.300'), (1, '42.820')] -[2023-10-15 16:17:58,617][52866] Updated weights for policy 1, policy_version 37690 (0.0008) -[2023-10-15 16:18:01,937][52833] Updated weights for policy 0, policy_version 37610 (0.0009) -[2023-10-15 16:18:02,290][52866] Updated weights for policy 1, policy_version 37700 (0.0007) -[2023-10-15 16:18:02,311][52833] Updated weights for policy 0, policy_version 37620 (0.0007) -[2023-10-15 16:18:02,653][52866] Updated weights for policy 1, policy_version 37710 (0.0007) -[2023-10-15 16:18:02,669][52833] Updated weights for policy 0, policy_version 37630 (0.0008) -[2023-10-15 16:18:03,025][52866] Updated weights for policy 1, policy_version 37720 (0.0010) -[2023-10-15 16:18:03,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 77168640. Throughput: 0: 1797.3, 1: 1780.9. Samples: 19290646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:18:03,441][51532] Avg episode reward: [(0, '42.440'), (1, '42.680')] -[2023-10-15 16:18:06,534][52833] Updated weights for policy 0, policy_version 37640 (0.0010) -[2023-10-15 16:18:06,793][52866] Updated weights for policy 1, policy_version 37730 (0.0009) -[2023-10-15 16:18:06,900][52833] Updated weights for policy 0, policy_version 37650 (0.0007) -[2023-10-15 16:18:07,152][52866] Updated weights for policy 1, policy_version 37740 (0.0007) -[2023-10-15 16:18:07,267][52833] Updated weights for policy 0, policy_version 37660 (0.0007) -[2023-10-15 16:18:07,521][52866] Updated weights for policy 1, policy_version 37750 (0.0007) -[2023-10-15 16:18:07,893][52866] Updated weights for policy 1, policy_version 37760 (0.0008) -[2023-10-15 16:18:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 77234176. Throughput: 0: 1807.4, 1: 1807.0. Samples: 19311736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:18:08,442][51532] Avg episode reward: [(0, '40.460'), (1, '44.280')] -[2023-10-15 16:18:10,851][52833] Updated weights for policy 0, policy_version 37670 (0.0008) -[2023-10-15 16:18:11,221][52833] Updated weights for policy 0, policy_version 37680 (0.0008) -[2023-10-15 16:18:11,587][52833] Updated weights for policy 0, policy_version 37690 (0.0008) -[2023-10-15 16:18:11,827][52866] Updated weights for policy 1, policy_version 37770 (0.0008) -[2023-10-15 16:18:12,192][52866] Updated weights for policy 1, policy_version 37780 (0.0008) -[2023-10-15 16:18:12,554][52866] Updated weights for policy 1, policy_version 37790 (0.0008) -[2023-10-15 16:18:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 77299712. Throughput: 0: 1797.2, 1: 1782.6. Samples: 19332470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:18:13,442][51532] Avg episode reward: [(0, '40.020'), (1, '46.150')] -[2023-10-15 16:18:15,431][52833] Updated weights for policy 0, policy_version 37700 (0.0008) -[2023-10-15 16:18:15,798][52833] Updated weights for policy 0, policy_version 37710 (0.0007) -[2023-10-15 16:18:16,162][52833] Updated weights for policy 0, policy_version 37720 (0.0008) -[2023-10-15 16:18:16,198][52866] Updated weights for policy 1, policy_version 37800 (0.0009) -[2023-10-15 16:18:16,562][52866] Updated weights for policy 1, policy_version 37810 (0.0008) -[2023-10-15 16:18:16,931][52866] Updated weights for policy 1, policy_version 37820 (0.0009) -[2023-10-15 16:18:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 77365248. Throughput: 0: 1809.6, 1: 1807.2. Samples: 19344290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:18:18,441][51532] Avg episode reward: [(0, '44.710'), (1, '45.460')] -[2023-10-15 16:18:19,929][52833] Updated weights for policy 0, policy_version 37730 (0.0008) -[2023-10-15 16:18:20,297][52833] Updated weights for policy 0, policy_version 37740 (0.0008) -[2023-10-15 16:18:20,656][52833] Updated weights for policy 0, policy_version 37750 (0.0007) -[2023-10-15 16:18:20,710][52866] Updated weights for policy 1, policy_version 37830 (0.0007) -[2023-10-15 16:18:21,024][52833] Updated weights for policy 0, policy_version 37760 (0.0009) -[2023-10-15 16:18:21,067][52866] Updated weights for policy 1, policy_version 37840 (0.0007) -[2023-10-15 16:18:21,439][52866] Updated weights for policy 1, policy_version 37850 (0.0009) -[2023-10-15 16:18:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 77430784. Throughput: 0: 1799.0, 1: 1787.7. Samples: 19364746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:18:23,441][51532] Avg episode reward: [(0, '44.340'), (1, '45.940')] -[2023-10-15 16:18:24,699][52833] Updated weights for policy 0, policy_version 37770 (0.0009) -[2023-10-15 16:18:25,067][52833] Updated weights for policy 0, policy_version 37780 (0.0010) -[2023-10-15 16:18:25,384][52866] Updated weights for policy 1, policy_version 37860 (0.0010) -[2023-10-15 16:18:25,435][52833] Updated weights for policy 0, policy_version 37790 (0.0008) -[2023-10-15 16:18:25,762][52866] Updated weights for policy 1, policy_version 37870 (0.0010) -[2023-10-15 16:18:26,130][52866] Updated weights for policy 1, policy_version 37880 (0.0009) -[2023-10-15 16:18:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 77496320. Throughput: 0: 1789.4, 1: 1780.8. Samples: 19386678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:18:28,442][51532] Avg episode reward: [(0, '44.000'), (1, '44.180')] -[2023-10-15 16:18:29,326][52833] Updated weights for policy 0, policy_version 37800 (0.0008) -[2023-10-15 16:18:29,694][52833] Updated weights for policy 0, policy_version 37810 (0.0009) -[2023-10-15 16:18:29,783][52866] Updated weights for policy 1, policy_version 37890 (0.0008) -[2023-10-15 16:18:30,063][52833] Updated weights for policy 0, policy_version 37820 (0.0007) -[2023-10-15 16:18:30,148][52866] Updated weights for policy 1, policy_version 37900 (0.0007) -[2023-10-15 16:18:30,518][52866] Updated weights for policy 1, policy_version 37910 (0.0009) -[2023-10-15 16:18:30,885][52866] Updated weights for policy 1, policy_version 37920 (0.0007) -[2023-10-15 16:18:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 77561856. Throughput: 0: 1791.1, 1: 1783.6. Samples: 19396674. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) -[2023-10-15 16:18:33,442][51532] Avg episode reward: [(0, '44.890'), (1, '42.470')] -[2023-10-15 16:18:33,836][52833] Updated weights for policy 0, policy_version 37830 (0.0008) -[2023-10-15 16:18:34,197][52833] Updated weights for policy 0, policy_version 37840 (0.0009) -[2023-10-15 16:18:34,568][52833] Updated weights for policy 0, policy_version 37850 (0.0007) -[2023-10-15 16:18:34,606][52866] Updated weights for policy 1, policy_version 37930 (0.0007) -[2023-10-15 16:18:34,968][52866] Updated weights for policy 1, policy_version 37940 (0.0010) -[2023-10-15 16:18:35,331][52866] Updated weights for policy 1, policy_version 37950 (0.0009) -[2023-10-15 16:18:38,336][52833] Updated weights for policy 0, policy_version 37860 (0.0008) -[2023-10-15 16:18:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 77627392. Throughput: 0: 1790.2, 1: 1781.5. Samples: 19419038. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) -[2023-10-15 16:18:38,442][51532] Avg episode reward: [(0, '42.570'), (1, '43.140')] -[2023-10-15 16:18:38,703][52833] Updated weights for policy 0, policy_version 37870 (0.0009) -[2023-10-15 16:18:39,064][52833] Updated weights for policy 0, policy_version 37880 (0.0008) -[2023-10-15 16:18:39,067][52866] Updated weights for policy 1, policy_version 37960 (0.0007) -[2023-10-15 16:18:39,437][52866] Updated weights for policy 1, policy_version 37970 (0.0008) -[2023-10-15 16:18:39,808][52866] Updated weights for policy 1, policy_version 37980 (0.0008) -[2023-10-15 16:18:42,794][52833] Updated weights for policy 0, policy_version 37890 (0.0009) -[2023-10-15 16:18:43,154][52833] Updated weights for policy 0, policy_version 37900 (0.0010) -[2023-10-15 16:18:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 77692928. Throughput: 0: 1811.5, 1: 1792.8. Samples: 19441378. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) -[2023-10-15 16:18:43,442][51532] Avg episode reward: [(0, '40.110'), (1, '43.290')] -[2023-10-15 16:18:43,514][52833] Updated weights for policy 0, policy_version 37910 (0.0009) -[2023-10-15 16:18:43,609][52866] Updated weights for policy 1, policy_version 37990 (0.0009) -[2023-10-15 16:18:43,888][52833] Updated weights for policy 0, policy_version 37920 (0.0009) -[2023-10-15 16:18:43,977][52866] Updated weights for policy 1, policy_version 38000 (0.0007) -[2023-10-15 16:18:44,352][52866] Updated weights for policy 1, policy_version 38010 (0.0008) -[2023-10-15 16:18:47,523][52833] Updated weights for policy 0, policy_version 37930 (0.0011) -[2023-10-15 16:18:47,899][52833] Updated weights for policy 0, policy_version 37940 (0.0011) -[2023-10-15 16:18:48,070][52866] Updated weights for policy 1, policy_version 38020 (0.0007) -[2023-10-15 16:18:48,263][52833] Updated weights for policy 0, policy_version 37950 (0.0007) -[2023-10-15 16:18:48,434][52866] Updated weights for policy 1, policy_version 38030 (0.0007) -[2023-10-15 16:18:48,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 77791232. Throughput: 0: 1791.6, 1: 1786.2. Samples: 19451650. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) -[2023-10-15 16:18:48,441][51532] Avg episode reward: [(0, '42.060'), (1, '43.020')] -[2023-10-15 16:18:48,801][52866] Updated weights for policy 1, policy_version 38040 (0.0008) -[2023-10-15 16:18:51,937][52833] Updated weights for policy 0, policy_version 37960 (0.0009) -[2023-10-15 16:18:52,301][52833] Updated weights for policy 0, policy_version 37970 (0.0011) -[2023-10-15 16:18:52,520][52866] Updated weights for policy 1, policy_version 38050 (0.0008) -[2023-10-15 16:18:52,676][52833] Updated weights for policy 0, policy_version 37980 (0.0008) -[2023-10-15 16:18:52,888][52866] Updated weights for policy 1, policy_version 38060 (0.0008) -[2023-10-15 16:18:53,260][52866] Updated weights for policy 1, policy_version 38070 (0.0010) -[2023-10-15 16:18:53,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 77856768. Throughput: 0: 1805.7, 1: 1791.0. Samples: 19473586. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) -[2023-10-15 16:18:53,442][51532] Avg episode reward: [(0, '41.520'), (1, '41.520')] -[2023-10-15 16:18:53,620][52866] Updated weights for policy 1, policy_version 38080 (0.0008) -[2023-10-15 16:18:56,411][52833] Updated weights for policy 0, policy_version 37990 (0.0008) -[2023-10-15 16:18:56,783][52833] Updated weights for policy 0, policy_version 38000 (0.0011) -[2023-10-15 16:18:57,157][52833] Updated weights for policy 0, policy_version 38010 (0.0010) -[2023-10-15 16:18:57,216][52866] Updated weights for policy 1, policy_version 38090 (0.0008) -[2023-10-15 16:18:57,585][52866] Updated weights for policy 1, policy_version 38100 (0.0009) -[2023-10-15 16:18:57,945][52866] Updated weights for policy 1, policy_version 38110 (0.0008) -[2023-10-15 16:18:58,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 77955072. Throughput: 0: 1787.2, 1: 1795.6. Samples: 19493696. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-15 16:18:58,442][51532] Avg episode reward: [(0, '44.280'), (1, '40.270')] -[2023-10-15 16:19:00,952][52833] Updated weights for policy 0, policy_version 38020 (0.0009) -[2023-10-15 16:19:01,320][52833] Updated weights for policy 0, policy_version 38030 (0.0009) -[2023-10-15 16:19:01,691][52833] Updated weights for policy 0, policy_version 38040 (0.0009) -[2023-10-15 16:19:01,695][52866] Updated weights for policy 1, policy_version 38120 (0.0009) -[2023-10-15 16:19:02,060][52866] Updated weights for policy 1, policy_version 38130 (0.0008) -[2023-10-15 16:19:02,425][52866] Updated weights for policy 1, policy_version 38140 (0.0008) -[2023-10-15 16:19:03,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 78020608. Throughput: 0: 1805.0, 1: 1795.5. Samples: 19506314. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-15 16:19:03,441][51532] Avg episode reward: [(0, '42.910'), (1, '41.430')] -[2023-10-15 16:19:05,362][52833] Updated weights for policy 0, policy_version 38050 (0.0008) -[2023-10-15 16:19:05,735][52833] Updated weights for policy 0, policy_version 38060 (0.0009) -[2023-10-15 16:19:06,102][52833] Updated weights for policy 0, policy_version 38070 (0.0009) -[2023-10-15 16:19:06,146][52866] Updated weights for policy 1, policy_version 38150 (0.0007) -[2023-10-15 16:19:06,471][52833] Updated weights for policy 0, policy_version 38080 (0.0009) -[2023-10-15 16:19:06,511][52866] Updated weights for policy 1, policy_version 38160 (0.0008) -[2023-10-15 16:19:06,878][52866] Updated weights for policy 1, policy_version 38170 (0.0008) -[2023-10-15 16:19:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 78086144. Throughput: 0: 1791.2, 1: 1798.2. Samples: 19526270. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-15 16:19:08,442][51532] Avg episode reward: [(0, '43.230'), (1, '43.350')] -[2023-10-15 16:19:10,406][52833] Updated weights for policy 0, policy_version 38090 (0.0008) -[2023-10-15 16:19:10,509][52866] Updated weights for policy 1, policy_version 38180 (0.0008) -[2023-10-15 16:19:10,770][52833] Updated weights for policy 0, policy_version 38100 (0.0008) -[2023-10-15 16:19:10,881][52866] Updated weights for policy 1, policy_version 38190 (0.0008) -[2023-10-15 16:19:11,144][52833] Updated weights for policy 0, policy_version 38110 (0.0009) -[2023-10-15 16:19:11,246][52866] Updated weights for policy 1, policy_version 38200 (0.0009) -[2023-10-15 16:19:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 78151680. Throughput: 0: 1797.8, 1: 1807.5. Samples: 19548918. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-15 16:19:13,442][51532] Avg episode reward: [(0, '42.470'), (1, '44.740')] -[2023-10-15 16:19:14,860][52866] Updated weights for policy 1, policy_version 38210 (0.0010) -[2023-10-15 16:19:14,893][52833] Updated weights for policy 0, policy_version 38120 (0.0009) -[2023-10-15 16:19:15,225][52866] Updated weights for policy 1, policy_version 38220 (0.0007) -[2023-10-15 16:19:15,269][52833] Updated weights for policy 0, policy_version 38130 (0.0008) -[2023-10-15 16:19:15,586][52866] Updated weights for policy 1, policy_version 38230 (0.0008) -[2023-10-15 16:19:15,641][52833] Updated weights for policy 0, policy_version 38140 (0.0008) -[2023-10-15 16:19:15,947][52866] Updated weights for policy 1, policy_version 38240 (0.0008) -[2023-10-15 16:19:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 78217216. Throughput: 0: 1793.1, 1: 1807.6. Samples: 19558706. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-15 16:19:18,442][51532] Avg episode reward: [(0, '44.930'), (1, '46.250')] -[2023-10-15 16:19:19,369][52833] Updated weights for policy 0, policy_version 38150 (0.0008) -[2023-10-15 16:19:19,733][52833] Updated weights for policy 0, policy_version 38160 (0.0008) -[2023-10-15 16:19:19,744][52866] Updated weights for policy 1, policy_version 38250 (0.0009) -[2023-10-15 16:19:20,110][52866] Updated weights for policy 1, policy_version 38260 (0.0007) -[2023-10-15 16:19:20,113][52833] Updated weights for policy 0, policy_version 38170 (0.0007) -[2023-10-15 16:19:20,470][52866] Updated weights for policy 1, policy_version 38270 (0.0010) -[2023-10-15 16:19:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 78282752. Throughput: 0: 1789.8, 1: 1811.2. Samples: 19581080. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-15 16:19:23,441][51532] Avg episode reward: [(0, '44.310'), (1, '43.960')] -[2023-10-15 16:19:23,893][52833] Updated weights for policy 0, policy_version 38180 (0.0008) -[2023-10-15 16:19:24,209][52866] Updated weights for policy 1, policy_version 38280 (0.0008) -[2023-10-15 16:19:24,257][52833] Updated weights for policy 0, policy_version 38190 (0.0008) -[2023-10-15 16:19:24,573][52866] Updated weights for policy 1, policy_version 38290 (0.0008) -[2023-10-15 16:19:24,627][52833] Updated weights for policy 0, policy_version 38200 (0.0009) -[2023-10-15 16:19:24,937][52866] Updated weights for policy 1, policy_version 38300 (0.0009) -[2023-10-15 16:19:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 78348288. Throughput: 0: 1790.1, 1: 1810.1. Samples: 19603390. Policy #0 lag: (min: 30.0, avg: 31.8, max: 58.0) -[2023-10-15 16:19:28,442][51532] Avg episode reward: [(0, '45.680'), (1, '44.490')] -[2023-10-15 16:19:28,471][52833] Updated weights for policy 0, policy_version 38210 (0.0008) -[2023-10-15 16:19:28,763][52866] Updated weights for policy 1, policy_version 38310 (0.0008) -[2023-10-15 16:19:28,846][52833] Updated weights for policy 0, policy_version 38220 (0.0009) -[2023-10-15 16:19:29,142][52866] Updated weights for policy 1, policy_version 38320 (0.0007) -[2023-10-15 16:19:29,211][52833] Updated weights for policy 0, policy_version 38230 (0.0008) -[2023-10-15 16:19:29,508][52866] Updated weights for policy 1, policy_version 38330 (0.0009) -[2023-10-15 16:19:29,577][52833] Updated weights for policy 0, policy_version 38240 (0.0008) -[2023-10-15 16:19:33,280][52866] Updated weights for policy 1, policy_version 38340 (0.0008) -[2023-10-15 16:19:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 78413824. Throughput: 0: 1783.0, 1: 1803.0. Samples: 19613020. Policy #0 lag: (min: 30.0, avg: 31.8, max: 58.0) -[2023-10-15 16:19:33,441][51532] Avg episode reward: [(0, '46.440'), (1, '44.420')] -[2023-10-15 16:19:33,536][52833] Updated weights for policy 0, policy_version 38250 (0.0009) -[2023-10-15 16:19:33,653][52866] Updated weights for policy 1, policy_version 38350 (0.0009) -[2023-10-15 16:19:33,905][52833] Updated weights for policy 0, policy_version 38260 (0.0007) -[2023-10-15 16:19:34,013][52866] Updated weights for policy 1, policy_version 38360 (0.0010) -[2023-10-15 16:19:34,270][52833] Updated weights for policy 0, policy_version 38270 (0.0008) -[2023-10-15 16:19:34,343][52410] Saving new best policy, reward=46.440! -[2023-10-15 16:19:37,825][52866] Updated weights for policy 1, policy_version 38370 (0.0007) -[2023-10-15 16:19:37,970][52833] Updated weights for policy 0, policy_version 38280 (0.0007) -[2023-10-15 16:19:38,192][52866] Updated weights for policy 1, policy_version 38380 (0.0007) -[2023-10-15 16:19:38,338][52833] Updated weights for policy 0, policy_version 38290 (0.0008) -[2023-10-15 16:19:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 78479360. Throughput: 0: 1790.7, 1: 1799.8. Samples: 19635158. Policy #0 lag: (min: 30.0, avg: 31.8, max: 58.0) -[2023-10-15 16:19:38,442][51532] Avg episode reward: [(0, '45.290'), (1, '43.840')] -[2023-10-15 16:19:38,552][52866] Updated weights for policy 1, policy_version 38390 (0.0008) -[2023-10-15 16:19:38,703][52833] Updated weights for policy 0, policy_version 38300 (0.0007) -[2023-10-15 16:19:38,924][52866] Updated weights for policy 1, policy_version 38400 (0.0009) -[2023-10-15 16:19:42,467][52833] Updated weights for policy 0, policy_version 38310 (0.0007) -[2023-10-15 16:19:42,609][52866] Updated weights for policy 1, policy_version 38410 (0.0008) -[2023-10-15 16:19:42,830][52833] Updated weights for policy 0, policy_version 38320 (0.0008) -[2023-10-15 16:19:42,972][52866] Updated weights for policy 1, policy_version 38420 (0.0009) -[2023-10-15 16:19:43,204][52833] Updated weights for policy 0, policy_version 38330 (0.0008) -[2023-10-15 16:19:43,339][52866] Updated weights for policy 1, policy_version 38430 (0.0007) -[2023-10-15 16:19:43,441][51532] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 78610432. Throughput: 0: 1800.7, 1: 1805.5. Samples: 19655972. Policy #0 lag: (min: 30.0, avg: 31.8, max: 58.0) -[2023-10-15 16:19:43,442][51532] Avg episode reward: [(0, '43.370'), (1, '45.360')] -[2023-10-15 16:19:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000038336_39256064.pth... -[2023-10-15 16:19:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000038432_39354368.pth... -[2023-10-15 16:19:43,480][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000036736_37617664.pth -[2023-10-15 16:19:43,489][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000036640_37519360.pth -[2023-10-15 16:19:46,970][52833] Updated weights for policy 0, policy_version 38340 (0.0007) -[2023-10-15 16:19:47,132][52866] Updated weights for policy 1, policy_version 38440 (0.0007) -[2023-10-15 16:19:47,341][52833] Updated weights for policy 0, policy_version 38350 (0.0007) -[2023-10-15 16:19:47,502][52866] Updated weights for policy 1, policy_version 38450 (0.0009) -[2023-10-15 16:19:47,707][52833] Updated weights for policy 0, policy_version 38360 (0.0008) -[2023-10-15 16:19:47,870][52866] Updated weights for policy 1, policy_version 38460 (0.0008) -[2023-10-15 16:19:48,441][51532] Fps is (10 sec: 19661.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 78675968. Throughput: 0: 1785.6, 1: 1792.7. Samples: 19667338. Policy #0 lag: (min: 30.0, avg: 31.8, max: 58.0) -[2023-10-15 16:19:48,441][51532] Avg episode reward: [(0, '42.230'), (1, '45.920')] -[2023-10-15 16:19:51,371][52833] Updated weights for policy 0, policy_version 38370 (0.0009) -[2023-10-15 16:19:51,743][52866] Updated weights for policy 1, policy_version 38470 (0.0008) -[2023-10-15 16:19:51,744][52833] Updated weights for policy 0, policy_version 38380 (0.0008) -[2023-10-15 16:19:52,106][52833] Updated weights for policy 0, policy_version 38390 (0.0008) -[2023-10-15 16:19:52,108][52866] Updated weights for policy 1, policy_version 38480 (0.0009) -[2023-10-15 16:19:52,476][52833] Updated weights for policy 0, policy_version 38400 (0.0008) -[2023-10-15 16:19:52,478][52866] Updated weights for policy 1, policy_version 38490 (0.0009) -[2023-10-15 16:19:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 78741504. Throughput: 0: 1797.0, 1: 1804.4. Samples: 19688332. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) -[2023-10-15 16:19:53,442][51532] Avg episode reward: [(0, '41.830'), (1, '43.470')] -[2023-10-15 16:19:56,118][52833] Updated weights for policy 0, policy_version 38410 (0.0009) -[2023-10-15 16:19:56,240][52866] Updated weights for policy 1, policy_version 38500 (0.0009) -[2023-10-15 16:19:56,483][52833] Updated weights for policy 0, policy_version 38420 (0.0008) -[2023-10-15 16:19:56,609][52866] Updated weights for policy 1, policy_version 38510 (0.0010) -[2023-10-15 16:19:56,848][52833] Updated weights for policy 0, policy_version 38430 (0.0007) -[2023-10-15 16:19:56,973][52866] Updated weights for policy 1, policy_version 38520 (0.0008) -[2023-10-15 16:19:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 78807040. Throughput: 0: 1785.2, 1: 1780.4. Samples: 19709366. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) -[2023-10-15 16:19:58,442][51532] Avg episode reward: [(0, '42.370'), (1, '40.590')] -[2023-10-15 16:20:00,547][52833] Updated weights for policy 0, policy_version 38440 (0.0008) -[2023-10-15 16:20:00,649][52866] Updated weights for policy 1, policy_version 38530 (0.0008) -[2023-10-15 16:20:00,909][52833] Updated weights for policy 0, policy_version 38450 (0.0008) -[2023-10-15 16:20:01,014][52866] Updated weights for policy 1, policy_version 38540 (0.0008) -[2023-10-15 16:20:01,274][52833] Updated weights for policy 0, policy_version 38460 (0.0009) -[2023-10-15 16:20:01,383][52866] Updated weights for policy 1, policy_version 38550 (0.0008) -[2023-10-15 16:20:01,751][52866] Updated weights for policy 1, policy_version 38560 (0.0007) -[2023-10-15 16:20:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 78872576. Throughput: 0: 1800.5, 1: 1801.0. Samples: 19720774. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) -[2023-10-15 16:20:03,442][51532] Avg episode reward: [(0, '42.630'), (1, '37.720')] -[2023-10-15 16:20:05,095][52833] Updated weights for policy 0, policy_version 38470 (0.0008) -[2023-10-15 16:20:05,459][52833] Updated weights for policy 0, policy_version 38480 (0.0008) -[2023-10-15 16:20:05,538][52866] Updated weights for policy 1, policy_version 38570 (0.0009) -[2023-10-15 16:20:05,834][52833] Updated weights for policy 0, policy_version 38490 (0.0009) -[2023-10-15 16:20:05,901][52866] Updated weights for policy 1, policy_version 38580 (0.0007) -[2023-10-15 16:20:06,268][52866] Updated weights for policy 1, policy_version 38590 (0.0008) -[2023-10-15 16:20:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 78938112. Throughput: 0: 1784.2, 1: 1778.3. Samples: 19741394. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) -[2023-10-15 16:20:08,442][51532] Avg episode reward: [(0, '41.360'), (1, '39.770')] -[2023-10-15 16:20:09,571][52833] Updated weights for policy 0, policy_version 38500 (0.0010) -[2023-10-15 16:20:09,943][52833] Updated weights for policy 0, policy_version 38510 (0.0008) -[2023-10-15 16:20:10,195][52866] Updated weights for policy 1, policy_version 38600 (0.0007) -[2023-10-15 16:20:10,316][52833] Updated weights for policy 0, policy_version 38520 (0.0009) -[2023-10-15 16:20:10,550][52866] Updated weights for policy 1, policy_version 38610 (0.0007) -[2023-10-15 16:20:10,927][52866] Updated weights for policy 1, policy_version 38620 (0.0008) -[2023-10-15 16:20:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 79003648. Throughput: 0: 1785.1, 1: 1779.3. Samples: 19763788. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) -[2023-10-15 16:20:13,442][51532] Avg episode reward: [(0, '40.190'), (1, '40.330')] -[2023-10-15 16:20:14,113][52833] Updated weights for policy 0, policy_version 38530 (0.0008) -[2023-10-15 16:20:14,478][52833] Updated weights for policy 0, policy_version 38540 (0.0008) -[2023-10-15 16:20:14,677][52866] Updated weights for policy 1, policy_version 38630 (0.0008) -[2023-10-15 16:20:14,847][52833] Updated weights for policy 0, policy_version 38550 (0.0009) -[2023-10-15 16:20:15,054][52866] Updated weights for policy 1, policy_version 38640 (0.0008) -[2023-10-15 16:20:15,221][52833] Updated weights for policy 0, policy_version 38560 (0.0008) -[2023-10-15 16:20:15,417][52866] Updated weights for policy 1, policy_version 38650 (0.0009) -[2023-10-15 16:20:18,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 79069184. Throughput: 0: 1782.6, 1: 1781.5. Samples: 19773406. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) -[2023-10-15 16:20:18,442][51532] Avg episode reward: [(0, '38.880'), (1, '41.440')] -[2023-10-15 16:20:19,039][52833] Updated weights for policy 0, policy_version 38570 (0.0007) -[2023-10-15 16:20:19,335][52866] Updated weights for policy 1, policy_version 38660 (0.0010) -[2023-10-15 16:20:19,412][52833] Updated weights for policy 0, policy_version 38580 (0.0007) -[2023-10-15 16:20:19,693][52866] Updated weights for policy 1, policy_version 38670 (0.0008) -[2023-10-15 16:20:19,781][52833] Updated weights for policy 0, policy_version 38590 (0.0009) -[2023-10-15 16:20:20,067][52866] Updated weights for policy 1, policy_version 38680 (0.0009) -[2023-10-15 16:20:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 79134720. Throughput: 0: 1782.9, 1: 1783.0. Samples: 19795624. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:20:23,442][51532] Avg episode reward: [(0, '40.880'), (1, '43.040')] -[2023-10-15 16:20:23,598][52833] Updated weights for policy 0, policy_version 38600 (0.0008) -[2023-10-15 16:20:23,886][52866] Updated weights for policy 1, policy_version 38690 (0.0009) -[2023-10-15 16:20:23,971][52833] Updated weights for policy 0, policy_version 38610 (0.0009) -[2023-10-15 16:20:24,261][52866] Updated weights for policy 1, policy_version 38700 (0.0008) -[2023-10-15 16:20:24,346][52833] Updated weights for policy 0, policy_version 38620 (0.0009) -[2023-10-15 16:20:24,627][52866] Updated weights for policy 1, policy_version 38710 (0.0009) -[2023-10-15 16:20:24,995][52866] Updated weights for policy 1, policy_version 38720 (0.0008) -[2023-10-15 16:20:28,015][52833] Updated weights for policy 0, policy_version 38630 (0.0007) -[2023-10-15 16:20:28,390][52833] Updated weights for policy 0, policy_version 38640 (0.0008) -[2023-10-15 16:20:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 79200256. Throughput: 0: 1799.4, 1: 1805.1. Samples: 19818172. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:20:28,441][51532] Avg episode reward: [(0, '41.360'), (1, '39.460')] -[2023-10-15 16:20:28,534][52866] Updated weights for policy 1, policy_version 38730 (0.0008) -[2023-10-15 16:20:28,759][52833] Updated weights for policy 0, policy_version 38650 (0.0009) -[2023-10-15 16:20:28,904][52866] Updated weights for policy 1, policy_version 38740 (0.0008) -[2023-10-15 16:20:29,266][52866] Updated weights for policy 1, policy_version 38750 (0.0009) -[2023-10-15 16:20:32,592][52833] Updated weights for policy 0, policy_version 38660 (0.0008) -[2023-10-15 16:20:32,964][52833] Updated weights for policy 0, policy_version 38670 (0.0009) -[2023-10-15 16:20:32,999][52866] Updated weights for policy 1, policy_version 38760 (0.0008) -[2023-10-15 16:20:33,333][52833] Updated weights for policy 0, policy_version 38680 (0.0008) -[2023-10-15 16:20:33,364][52866] Updated weights for policy 1, policy_version 38770 (0.0009) -[2023-10-15 16:20:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 79265792. Throughput: 0: 1785.2, 1: 1789.7. Samples: 19828206. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:20:33,441][51532] Avg episode reward: [(0, '42.420'), (1, '40.450')] -[2023-10-15 16:20:33,726][52866] Updated weights for policy 1, policy_version 38780 (0.0008) -[2023-10-15 16:20:37,259][52833] Updated weights for policy 0, policy_version 38690 (0.0008) -[2023-10-15 16:20:37,640][52833] Updated weights for policy 0, policy_version 38700 (0.0009) -[2023-10-15 16:20:37,753][52866] Updated weights for policy 1, policy_version 38790 (0.0009) -[2023-10-15 16:20:38,019][52833] Updated weights for policy 0, policy_version 38710 (0.0008) -[2023-10-15 16:20:38,120][52866] Updated weights for policy 1, policy_version 38800 (0.0008) -[2023-10-15 16:20:38,391][52833] Updated weights for policy 0, policy_version 38720 (0.0009) -[2023-10-15 16:20:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 79364096. Throughput: 0: 1799.2, 1: 1800.1. Samples: 19850296. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:20:38,441][51532] Avg episode reward: [(0, '42.780'), (1, '38.220')] -[2023-10-15 16:20:38,487][52866] Updated weights for policy 1, policy_version 38810 (0.0008) -[2023-10-15 16:20:42,202][52833] Updated weights for policy 0, policy_version 38730 (0.0009) -[2023-10-15 16:20:42,240][52866] Updated weights for policy 1, policy_version 38820 (0.0008) -[2023-10-15 16:20:42,568][52833] Updated weights for policy 0, policy_version 38740 (0.0007) -[2023-10-15 16:20:42,601][52866] Updated weights for policy 1, policy_version 38830 (0.0007) -[2023-10-15 16:20:42,929][52833] Updated weights for policy 0, policy_version 38750 (0.0007) -[2023-10-15 16:20:42,962][52866] Updated weights for policy 1, policy_version 38840 (0.0008) -[2023-10-15 16:20:43,441][51532] Fps is (10 sec: 19660.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 79462400. Throughput: 0: 1777.2, 1: 1798.0. Samples: 19870248. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:20:43,442][51532] Avg episode reward: [(0, '42.210'), (1, '38.940')] -[2023-10-15 16:20:46,534][52833] Updated weights for policy 0, policy_version 38760 (0.0010) -[2023-10-15 16:20:46,754][52866] Updated weights for policy 1, policy_version 38850 (0.0009) -[2023-10-15 16:20:46,901][52833] Updated weights for policy 0, policy_version 38770 (0.0008) -[2023-10-15 16:20:47,122][52866] Updated weights for policy 1, policy_version 38860 (0.0010) -[2023-10-15 16:20:47,269][52833] Updated weights for policy 0, policy_version 38780 (0.0008) -[2023-10-15 16:20:47,488][52866] Updated weights for policy 1, policy_version 38870 (0.0009) -[2023-10-15 16:20:47,848][52866] Updated weights for policy 1, policy_version 38880 (0.0010) -[2023-10-15 16:20:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 79527936. Throughput: 0: 1794.4, 1: 1794.2. Samples: 19882262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:20:48,442][51532] Avg episode reward: [(0, '43.320'), (1, '38.640')] -[2023-10-15 16:20:51,096][52833] Updated weights for policy 0, policy_version 38790 (0.0009) -[2023-10-15 16:20:51,462][52833] Updated weights for policy 0, policy_version 38800 (0.0007) -[2023-10-15 16:20:51,567][52866] Updated weights for policy 1, policy_version 38890 (0.0008) -[2023-10-15 16:20:51,833][52833] Updated weights for policy 0, policy_version 38810 (0.0007) -[2023-10-15 16:20:51,938][52866] Updated weights for policy 1, policy_version 38900 (0.0008) -[2023-10-15 16:20:52,297][52866] Updated weights for policy 1, policy_version 38910 (0.0007) -[2023-10-15 16:20:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 79593472. Throughput: 0: 1783.1, 1: 1793.8. Samples: 19902352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:20:53,442][51532] Avg episode reward: [(0, '43.270'), (1, '39.410')] -[2023-10-15 16:20:55,692][52833] Updated weights for policy 0, policy_version 38820 (0.0007) -[2023-10-15 16:20:56,054][52833] Updated weights for policy 0, policy_version 38830 (0.0008) -[2023-10-15 16:20:56,158][52866] Updated weights for policy 1, policy_version 38920 (0.0008) -[2023-10-15 16:20:56,433][52833] Updated weights for policy 0, policy_version 38840 (0.0008) -[2023-10-15 16:20:56,521][52866] Updated weights for policy 1, policy_version 38930 (0.0007) -[2023-10-15 16:20:56,894][52866] Updated weights for policy 1, policy_version 38940 (0.0009) -[2023-10-15 16:20:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 79659008. Throughput: 0: 1776.9, 1: 1776.0. Samples: 19923670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:20:58,442][51532] Avg episode reward: [(0, '46.160'), (1, '38.970')] -[2023-10-15 16:21:00,219][52833] Updated weights for policy 0, policy_version 38850 (0.0008) -[2023-10-15 16:21:00,590][52833] Updated weights for policy 0, policy_version 38860 (0.0008) -[2023-10-15 16:21:00,725][52866] Updated weights for policy 1, policy_version 38950 (0.0008) -[2023-10-15 16:21:00,957][52833] Updated weights for policy 0, policy_version 38870 (0.0007) -[2023-10-15 16:21:01,091][52866] Updated weights for policy 1, policy_version 38960 (0.0008) -[2023-10-15 16:21:01,321][52833] Updated weights for policy 0, policy_version 38880 (0.0008) -[2023-10-15 16:21:01,464][52866] Updated weights for policy 1, policy_version 38970 (0.0008) -[2023-10-15 16:21:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 79724544. Throughput: 0: 1786.2, 1: 1795.9. Samples: 19934600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:21:03,442][51532] Avg episode reward: [(0, '47.640'), (1, '36.520')] -[2023-10-15 16:21:03,442][52410] Saving new best policy, reward=47.640! -[2023-10-15 16:21:05,109][52833] Updated weights for policy 0, policy_version 38890 (0.0007) -[2023-10-15 16:21:05,218][52866] Updated weights for policy 1, policy_version 38980 (0.0009) -[2023-10-15 16:21:05,484][52833] Updated weights for policy 0, policy_version 38900 (0.0007) -[2023-10-15 16:21:05,583][52866] Updated weights for policy 1, policy_version 38990 (0.0008) -[2023-10-15 16:21:05,845][52833] Updated weights for policy 0, policy_version 38910 (0.0008) -[2023-10-15 16:21:05,951][52866] Updated weights for policy 1, policy_version 39000 (0.0007) -[2023-10-15 16:21:08,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 79790080. Throughput: 0: 1772.9, 1: 1775.7. Samples: 19955312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:21:08,441][51532] Avg episode reward: [(0, '45.850'), (1, '37.570')] -[2023-10-15 16:21:09,703][52833] Updated weights for policy 0, policy_version 38920 (0.0008) -[2023-10-15 16:21:09,758][52866] Updated weights for policy 1, policy_version 39010 (0.0008) -[2023-10-15 16:21:10,066][52833] Updated weights for policy 0, policy_version 38930 (0.0008) -[2023-10-15 16:21:10,124][52866] Updated weights for policy 1, policy_version 39020 (0.0007) -[2023-10-15 16:21:10,432][52833] Updated weights for policy 0, policy_version 38940 (0.0007) -[2023-10-15 16:21:10,496][52866] Updated weights for policy 1, policy_version 39030 (0.0007) -[2023-10-15 16:21:10,870][52866] Updated weights for policy 1, policy_version 39040 (0.0010) -[2023-10-15 16:21:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 79855616. Throughput: 0: 1772.8, 1: 1769.6. Samples: 19977580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:21:13,442][51532] Avg episode reward: [(0, '45.210'), (1, '36.140')] -[2023-10-15 16:21:14,196][52833] Updated weights for policy 0, policy_version 38950 (0.0008) -[2023-10-15 16:21:14,569][52833] Updated weights for policy 0, policy_version 38960 (0.0008) -[2023-10-15 16:21:14,671][52866] Updated weights for policy 1, policy_version 39050 (0.0008) -[2023-10-15 16:21:14,931][52833] Updated weights for policy 0, policy_version 38970 (0.0007) -[2023-10-15 16:21:15,032][52866] Updated weights for policy 1, policy_version 39060 (0.0007) -[2023-10-15 16:21:15,390][52866] Updated weights for policy 1, policy_version 39070 (0.0008) -[2023-10-15 16:21:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 79921152. Throughput: 0: 1771.0, 1: 1767.2. Samples: 19987424. Policy #0 lag: (min: 35.0, avg: 47.1, max: 48.0) -[2023-10-15 16:21:18,442][51532] Avg episode reward: [(0, '43.460'), (1, '37.910')] -[2023-10-15 16:21:18,710][52833] Updated weights for policy 0, policy_version 38980 (0.0008) -[2023-10-15 16:21:18,984][52866] Updated weights for policy 1, policy_version 39080 (0.0008) -[2023-10-15 16:21:19,087][52833] Updated weights for policy 0, policy_version 38990 (0.0009) -[2023-10-15 16:21:19,346][52866] Updated weights for policy 1, policy_version 39090 (0.0007) -[2023-10-15 16:21:19,464][52833] Updated weights for policy 0, policy_version 39000 (0.0008) -[2023-10-15 16:21:19,712][52866] Updated weights for policy 1, policy_version 39100 (0.0008) -[2023-10-15 16:21:23,120][52833] Updated weights for policy 0, policy_version 39010 (0.0008) -[2023-10-15 16:21:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 79986688. Throughput: 0: 1768.0, 1: 1771.6. Samples: 20009576. Policy #0 lag: (min: 35.0, avg: 47.1, max: 48.0) -[2023-10-15 16:21:23,441][51532] Avg episode reward: [(0, '43.490'), (1, '39.810')] -[2023-10-15 16:21:23,479][52833] Updated weights for policy 0, policy_version 39020 (0.0009) -[2023-10-15 16:21:23,549][52866] Updated weights for policy 1, policy_version 39110 (0.0008) -[2023-10-15 16:21:23,846][52833] Updated weights for policy 0, policy_version 39030 (0.0008) -[2023-10-15 16:21:23,919][52866] Updated weights for policy 1, policy_version 39120 (0.0008) -[2023-10-15 16:21:24,216][52833] Updated weights for policy 0, policy_version 39040 (0.0009) -[2023-10-15 16:21:24,278][52866] Updated weights for policy 1, policy_version 39130 (0.0008) -[2023-10-15 16:21:27,954][52833] Updated weights for policy 0, policy_version 39050 (0.0009) -[2023-10-15 16:21:28,055][52866] Updated weights for policy 1, policy_version 39140 (0.0009) -[2023-10-15 16:21:28,327][52833] Updated weights for policy 0, policy_version 39060 (0.0009) -[2023-10-15 16:21:28,423][52866] Updated weights for policy 1, policy_version 39150 (0.0007) -[2023-10-15 16:21:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 80052224. Throughput: 0: 1792.9, 1: 1794.5. Samples: 20031676. Policy #0 lag: (min: 35.0, avg: 47.1, max: 48.0) -[2023-10-15 16:21:28,441][51532] Avg episode reward: [(0, '45.330'), (1, '40.790')] -[2023-10-15 16:21:28,693][52833] Updated weights for policy 0, policy_version 39070 (0.0009) -[2023-10-15 16:21:28,783][52866] Updated weights for policy 1, policy_version 39160 (0.0008) -[2023-10-15 16:21:32,492][52833] Updated weights for policy 0, policy_version 39080 (0.0008) -[2023-10-15 16:21:32,592][52866] Updated weights for policy 1, policy_version 39170 (0.0008) -[2023-10-15 16:21:32,872][52833] Updated weights for policy 0, policy_version 39090 (0.0009) -[2023-10-15 16:21:32,950][52866] Updated weights for policy 1, policy_version 39180 (0.0007) -[2023-10-15 16:21:33,245][52833] Updated weights for policy 0, policy_version 39100 (0.0009) -[2023-10-15 16:21:33,322][52866] Updated weights for policy 1, policy_version 39190 (0.0009) -[2023-10-15 16:21:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 80150528. Throughput: 0: 1772.2, 1: 1773.9. Samples: 20041834. Policy #0 lag: (min: 35.0, avg: 47.1, max: 48.0) -[2023-10-15 16:21:33,441][51532] Avg episode reward: [(0, '43.930'), (1, '38.650')] -[2023-10-15 16:21:33,680][52866] Updated weights for policy 1, policy_version 39200 (0.0010) -[2023-10-15 16:21:36,988][52833] Updated weights for policy 0, policy_version 39110 (0.0009) -[2023-10-15 16:21:37,360][52833] Updated weights for policy 0, policy_version 39120 (0.0009) -[2023-10-15 16:21:37,461][52866] Updated weights for policy 1, policy_version 39210 (0.0007) -[2023-10-15 16:21:37,726][52833] Updated weights for policy 0, policy_version 39130 (0.0009) -[2023-10-15 16:21:37,827][52866] Updated weights for policy 1, policy_version 39220 (0.0007) -[2023-10-15 16:21:38,204][52866] Updated weights for policy 1, policy_version 39230 (0.0007) -[2023-10-15 16:21:38,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 80248832. Throughput: 0: 1798.4, 1: 1790.2. Samples: 20063836. Policy #0 lag: (min: 35.0, avg: 47.1, max: 48.0) -[2023-10-15 16:21:38,441][51532] Avg episode reward: [(0, '40.010'), (1, '36.210')] -[2023-10-15 16:21:41,607][52833] Updated weights for policy 0, policy_version 39140 (0.0009) -[2023-10-15 16:21:41,972][52833] Updated weights for policy 0, policy_version 39150 (0.0007) -[2023-10-15 16:21:42,007][52866] Updated weights for policy 1, policy_version 39240 (0.0007) -[2023-10-15 16:21:42,341][52833] Updated weights for policy 0, policy_version 39160 (0.0008) -[2023-10-15 16:21:42,376][52866] Updated weights for policy 1, policy_version 39250 (0.0007) -[2023-10-15 16:21:42,753][52866] Updated weights for policy 1, policy_version 39260 (0.0007) -[2023-10-15 16:21:43,441][51532] Fps is (10 sec: 16383.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 80314368. Throughput: 0: 1772.4, 1: 1779.8. Samples: 20083516. Policy #0 lag: (min: 16.0, avg: 41.0, max: 48.0) -[2023-10-15 16:21:43,442][51532] Avg episode reward: [(0, '38.420'), (1, '34.330')] -[2023-10-15 16:21:43,455][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000039264_40206336.pth... -[2023-10-15 16:21:43,455][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000039168_40108032.pth... -[2023-10-15 16:21:43,508][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000037568_38469632.pth -[2023-10-15 16:21:43,508][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000037504_38404096.pth -[2023-10-15 16:21:46,139][52833] Updated weights for policy 0, policy_version 39170 (0.0008) -[2023-10-15 16:21:46,509][52833] Updated weights for policy 0, policy_version 39180 (0.0009) -[2023-10-15 16:21:46,594][52866] Updated weights for policy 1, policy_version 39270 (0.0008) -[2023-10-15 16:21:46,874][52833] Updated weights for policy 0, policy_version 39190 (0.0008) -[2023-10-15 16:21:46,961][52866] Updated weights for policy 1, policy_version 39280 (0.0009) -[2023-10-15 16:21:47,245][52833] Updated weights for policy 0, policy_version 39200 (0.0008) -[2023-10-15 16:21:47,334][52866] Updated weights for policy 1, policy_version 39290 (0.0010) -[2023-10-15 16:21:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 80379904. Throughput: 0: 1798.3, 1: 1793.1. Samples: 20096214. Policy #0 lag: (min: 16.0, avg: 41.0, max: 48.0) -[2023-10-15 16:21:48,442][51532] Avg episode reward: [(0, '39.120'), (1, '35.710')] -[2023-10-15 16:21:50,938][52833] Updated weights for policy 0, policy_version 39210 (0.0007) -[2023-10-15 16:21:51,192][52866] Updated weights for policy 1, policy_version 39300 (0.0009) -[2023-10-15 16:21:51,305][52833] Updated weights for policy 0, policy_version 39220 (0.0008) -[2023-10-15 16:21:51,559][52866] Updated weights for policy 1, policy_version 39310 (0.0009) -[2023-10-15 16:21:51,675][52833] Updated weights for policy 0, policy_version 39230 (0.0007) -[2023-10-15 16:21:51,922][52866] Updated weights for policy 1, policy_version 39320 (0.0009) -[2023-10-15 16:21:53,441][51532] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 80445440. Throughput: 0: 1783.7, 1: 1787.4. Samples: 20116010. Policy #0 lag: (min: 16.0, avg: 41.0, max: 48.0) -[2023-10-15 16:21:53,441][51532] Avg episode reward: [(0, '41.920'), (1, '36.400')] -[2023-10-15 16:21:55,494][52833] Updated weights for policy 0, policy_version 39240 (0.0009) -[2023-10-15 16:21:55,648][52866] Updated weights for policy 1, policy_version 39330 (0.0009) -[2023-10-15 16:21:55,869][52833] Updated weights for policy 0, policy_version 39250 (0.0009) -[2023-10-15 16:21:56,010][52866] Updated weights for policy 1, policy_version 39340 (0.0009) -[2023-10-15 16:21:56,248][52833] Updated weights for policy 0, policy_version 39260 (0.0008) -[2023-10-15 16:21:56,381][52866] Updated weights for policy 1, policy_version 39350 (0.0009) -[2023-10-15 16:21:56,754][52866] Updated weights for policy 1, policy_version 39360 (0.0009) -[2023-10-15 16:21:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 80510976. Throughput: 0: 1782.8, 1: 1785.0. Samples: 20138130. Policy #0 lag: (min: 16.0, avg: 41.0, max: 48.0) -[2023-10-15 16:21:58,442][51532] Avg episode reward: [(0, '41.160'), (1, '35.520')] -[2023-10-15 16:21:59,962][52833] Updated weights for policy 0, policy_version 39270 (0.0007) -[2023-10-15 16:22:00,331][52833] Updated weights for policy 0, policy_version 39280 (0.0010) -[2023-10-15 16:22:00,524][52866] Updated weights for policy 1, policy_version 39370 (0.0008) -[2023-10-15 16:22:00,698][52833] Updated weights for policy 0, policy_version 39290 (0.0008) -[2023-10-15 16:22:00,896][52866] Updated weights for policy 1, policy_version 39380 (0.0009) -[2023-10-15 16:22:01,263][52866] Updated weights for policy 1, policy_version 39390 (0.0008) -[2023-10-15 16:22:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 80576512. Throughput: 0: 1785.5, 1: 1794.6. Samples: 20148528. Policy #0 lag: (min: 16.0, avg: 41.0, max: 48.0) -[2023-10-15 16:22:03,442][51532] Avg episode reward: [(0, '40.380'), (1, '36.390')] -[2023-10-15 16:22:04,394][52833] Updated weights for policy 0, policy_version 39300 (0.0007) -[2023-10-15 16:22:04,762][52833] Updated weights for policy 0, policy_version 39310 (0.0009) -[2023-10-15 16:22:04,971][52866] Updated weights for policy 1, policy_version 39400 (0.0007) -[2023-10-15 16:22:05,130][52833] Updated weights for policy 0, policy_version 39320 (0.0009) -[2023-10-15 16:22:05,339][52866] Updated weights for policy 1, policy_version 39410 (0.0007) -[2023-10-15 16:22:05,703][52866] Updated weights for policy 1, policy_version 39420 (0.0008) -[2023-10-15 16:22:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 80642048. Throughput: 0: 1785.6, 1: 1779.6. Samples: 20170008. Policy #0 lag: (min: 16.0, avg: 41.0, max: 48.0) -[2023-10-15 16:22:08,441][51532] Avg episode reward: [(0, '38.680'), (1, '37.810')] -[2023-10-15 16:22:09,069][52833] Updated weights for policy 0, policy_version 39330 (0.0007) -[2023-10-15 16:22:09,438][52833] Updated weights for policy 0, policy_version 39340 (0.0007) -[2023-10-15 16:22:09,476][52866] Updated weights for policy 1, policy_version 39430 (0.0009) -[2023-10-15 16:22:09,802][52833] Updated weights for policy 0, policy_version 39350 (0.0009) -[2023-10-15 16:22:09,841][52866] Updated weights for policy 1, policy_version 39440 (0.0010) -[2023-10-15 16:22:10,170][52833] Updated weights for policy 0, policy_version 39360 (0.0009) -[2023-10-15 16:22:10,210][52866] Updated weights for policy 1, policy_version 39450 (0.0008) -[2023-10-15 16:22:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 80707584. Throughput: 0: 1785.5, 1: 1783.0. Samples: 20192256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:22:13,442][51532] Avg episode reward: [(0, '37.730'), (1, '40.100')] -[2023-10-15 16:22:13,948][52866] Updated weights for policy 1, policy_version 39460 (0.0007) -[2023-10-15 16:22:14,028][52833] Updated weights for policy 0, policy_version 39370 (0.0008) -[2023-10-15 16:22:14,309][52866] Updated weights for policy 1, policy_version 39470 (0.0009) -[2023-10-15 16:22:14,391][52833] Updated weights for policy 0, policy_version 39380 (0.0008) -[2023-10-15 16:22:14,671][52866] Updated weights for policy 1, policy_version 39480 (0.0008) -[2023-10-15 16:22:14,766][52833] Updated weights for policy 0, policy_version 39390 (0.0008) -[2023-10-15 16:22:18,388][52866] Updated weights for policy 1, policy_version 39490 (0.0008) -[2023-10-15 16:22:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 80773120. Throughput: 0: 1773.6, 1: 1786.3. Samples: 20202032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:22:18,441][51532] Avg episode reward: [(0, '39.110'), (1, '43.650')] -[2023-10-15 16:22:18,464][52833] Updated weights for policy 0, policy_version 39400 (0.0008) -[2023-10-15 16:22:18,751][52866] Updated weights for policy 1, policy_version 39500 (0.0007) -[2023-10-15 16:22:18,824][52833] Updated weights for policy 0, policy_version 39410 (0.0009) -[2023-10-15 16:22:19,109][52866] Updated weights for policy 1, policy_version 39510 (0.0007) -[2023-10-15 16:22:19,194][52833] Updated weights for policy 0, policy_version 39420 (0.0009) -[2023-10-15 16:22:19,471][52866] Updated weights for policy 1, policy_version 39520 (0.0009) -[2023-10-15 16:22:22,994][52833] Updated weights for policy 0, policy_version 39430 (0.0010) -[2023-10-15 16:22:23,133][52866] Updated weights for policy 1, policy_version 39530 (0.0008) -[2023-10-15 16:22:23,355][52833] Updated weights for policy 0, policy_version 39440 (0.0007) -[2023-10-15 16:22:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 80838656. Throughput: 0: 1775.8, 1: 1798.3. Samples: 20224670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:22:23,442][51532] Avg episode reward: [(0, '40.760'), (1, '45.710')] -[2023-10-15 16:22:23,507][52866] Updated weights for policy 1, policy_version 39540 (0.0008) -[2023-10-15 16:22:23,725][52833] Updated weights for policy 0, policy_version 39450 (0.0010) -[2023-10-15 16:22:23,873][52866] Updated weights for policy 1, policy_version 39550 (0.0007) -[2023-10-15 16:22:27,540][52866] Updated weights for policy 1, policy_version 39560 (0.0010) -[2023-10-15 16:22:27,627][52833] Updated weights for policy 0, policy_version 39460 (0.0009) -[2023-10-15 16:22:27,905][52866] Updated weights for policy 1, policy_version 39570 (0.0009) -[2023-10-15 16:22:27,992][52833] Updated weights for policy 0, policy_version 39470 (0.0009) -[2023-10-15 16:22:28,269][52866] Updated weights for policy 1, policy_version 39580 (0.0008) -[2023-10-15 16:22:28,353][52833] Updated weights for policy 0, policy_version 39480 (0.0008) -[2023-10-15 16:22:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 80936960. Throughput: 0: 1795.9, 1: 1812.8. Samples: 20245908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:22:28,441][51532] Avg episode reward: [(0, '42.480'), (1, '44.950')] -[2023-10-15 16:22:31,924][52866] Updated weights for policy 1, policy_version 39590 (0.0007) -[2023-10-15 16:22:32,112][52833] Updated weights for policy 0, policy_version 39490 (0.0008) -[2023-10-15 16:22:32,295][52866] Updated weights for policy 1, policy_version 39600 (0.0007) -[2023-10-15 16:22:32,481][52833] Updated weights for policy 0, policy_version 39500 (0.0008) -[2023-10-15 16:22:32,655][52866] Updated weights for policy 1, policy_version 39610 (0.0007) -[2023-10-15 16:22:32,857][52833] Updated weights for policy 0, policy_version 39510 (0.0007) -[2023-10-15 16:22:33,227][52833] Updated weights for policy 0, policy_version 39520 (0.0008) -[2023-10-15 16:22:33,441][51532] Fps is (10 sec: 19661.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 81035264. Throughput: 0: 1774.8, 1: 1806.9. Samples: 20257390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:22:33,442][51532] Avg episode reward: [(0, '39.710'), (1, '46.490')] -[2023-10-15 16:22:36,327][52866] Updated weights for policy 1, policy_version 39620 (0.0008) -[2023-10-15 16:22:36,701][52866] Updated weights for policy 1, policy_version 39630 (0.0008) -[2023-10-15 16:22:37,062][52866] Updated weights for policy 1, policy_version 39640 (0.0009) -[2023-10-15 16:22:37,093][52833] Updated weights for policy 0, policy_version 39530 (0.0009) -[2023-10-15 16:22:37,465][52833] Updated weights for policy 0, policy_version 39540 (0.0007) -[2023-10-15 16:22:37,838][52833] Updated weights for policy 0, policy_version 39550 (0.0009) -[2023-10-15 16:22:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 81100800. Throughput: 0: 1796.0, 1: 1813.8. Samples: 20278450. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) -[2023-10-15 16:22:38,441][51532] Avg episode reward: [(0, '39.440'), (1, '49.220')] -[2023-10-15 16:22:38,442][52518] Saving new best policy, reward=49.220! -[2023-10-15 16:22:40,807][52866] Updated weights for policy 1, policy_version 39650 (0.0009) -[2023-10-15 16:22:41,174][52866] Updated weights for policy 1, policy_version 39660 (0.0008) -[2023-10-15 16:22:41,530][52866] Updated weights for policy 1, policy_version 39670 (0.0008) -[2023-10-15 16:22:41,714][52833] Updated weights for policy 0, policy_version 39560 (0.0008) -[2023-10-15 16:22:41,901][52866] Updated weights for policy 1, policy_version 39680 (0.0007) -[2023-10-15 16:22:42,076][52833] Updated weights for policy 0, policy_version 39570 (0.0008) -[2023-10-15 16:22:42,454][52833] Updated weights for policy 0, policy_version 39580 (0.0008) -[2023-10-15 16:22:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81166336. Throughput: 0: 1768.8, 1: 1807.8. Samples: 20299074. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) -[2023-10-15 16:22:43,442][51532] Avg episode reward: [(0, '37.440'), (1, '48.050')] -[2023-10-15 16:22:45,684][52866] Updated weights for policy 1, policy_version 39690 (0.0009) -[2023-10-15 16:22:46,054][52866] Updated weights for policy 1, policy_version 39700 (0.0009) -[2023-10-15 16:22:46,152][52833] Updated weights for policy 0, policy_version 39590 (0.0007) -[2023-10-15 16:22:46,426][52866] Updated weights for policy 1, policy_version 39710 (0.0009) -[2023-10-15 16:22:46,522][52833] Updated weights for policy 0, policy_version 39600 (0.0007) -[2023-10-15 16:22:46,892][52833] Updated weights for policy 0, policy_version 39610 (0.0010) -[2023-10-15 16:22:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81231872. Throughput: 0: 1800.4, 1: 1810.9. Samples: 20311036. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) -[2023-10-15 16:22:48,441][51532] Avg episode reward: [(0, '38.630'), (1, '47.190')] -[2023-10-15 16:22:50,128][52866] Updated weights for policy 1, policy_version 39720 (0.0008) -[2023-10-15 16:22:50,495][52866] Updated weights for policy 1, policy_version 39730 (0.0008) -[2023-10-15 16:22:50,699][52833] Updated weights for policy 0, policy_version 39620 (0.0010) -[2023-10-15 16:22:50,852][52866] Updated weights for policy 1, policy_version 39740 (0.0008) -[2023-10-15 16:22:51,066][52833] Updated weights for policy 0, policy_version 39630 (0.0007) -[2023-10-15 16:22:51,435][52833] Updated weights for policy 0, policy_version 39640 (0.0010) -[2023-10-15 16:22:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 81297408. Throughput: 0: 1769.2, 1: 1814.2. Samples: 20331260. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) -[2023-10-15 16:22:53,441][51532] Avg episode reward: [(0, '40.520'), (1, '48.850')] -[2023-10-15 16:22:54,574][52866] Updated weights for policy 1, policy_version 39750 (0.0009) -[2023-10-15 16:22:54,938][52866] Updated weights for policy 1, policy_version 39760 (0.0009) -[2023-10-15 16:22:55,136][52833] Updated weights for policy 0, policy_version 39650 (0.0007) -[2023-10-15 16:22:55,303][52866] Updated weights for policy 1, policy_version 39770 (0.0007) -[2023-10-15 16:22:55,499][52833] Updated weights for policy 0, policy_version 39660 (0.0009) -[2023-10-15 16:22:55,876][52833] Updated weights for policy 0, policy_version 39670 (0.0007) -[2023-10-15 16:22:56,249][52833] Updated weights for policy 0, policy_version 39680 (0.0009) -[2023-10-15 16:22:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 81362944. Throughput: 0: 1772.3, 1: 1817.6. Samples: 20353800. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) -[2023-10-15 16:22:58,441][51532] Avg episode reward: [(0, '36.430'), (1, '47.770')] -[2023-10-15 16:22:58,944][52866] Updated weights for policy 1, policy_version 39780 (0.0008) -[2023-10-15 16:22:59,302][52866] Updated weights for policy 1, policy_version 39790 (0.0007) -[2023-10-15 16:22:59,667][52866] Updated weights for policy 1, policy_version 39800 (0.0009) -[2023-10-15 16:22:59,995][52833] Updated weights for policy 0, policy_version 39690 (0.0008) -[2023-10-15 16:23:00,360][52833] Updated weights for policy 0, policy_version 39700 (0.0009) -[2023-10-15 16:23:00,732][52833] Updated weights for policy 0, policy_version 39710 (0.0010) -[2023-10-15 16:23:03,215][52866] Updated weights for policy 1, policy_version 39810 (0.0008) -[2023-10-15 16:23:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 81428480. Throughput: 0: 1773.7, 1: 1817.2. Samples: 20363626. Policy #0 lag: (min: 14.0, avg: 21.9, max: 46.0) -[2023-10-15 16:23:03,442][51532] Avg episode reward: [(0, '36.330'), (1, '45.850')] -[2023-10-15 16:23:03,577][52866] Updated weights for policy 1, policy_version 39820 (0.0008) -[2023-10-15 16:23:03,947][52866] Updated weights for policy 1, policy_version 39830 (0.0008) -[2023-10-15 16:23:04,321][52866] Updated weights for policy 1, policy_version 39840 (0.0008) -[2023-10-15 16:23:04,453][52833] Updated weights for policy 0, policy_version 39720 (0.0009) -[2023-10-15 16:23:04,830][52833] Updated weights for policy 0, policy_version 39730 (0.0008) -[2023-10-15 16:23:05,194][52833] Updated weights for policy 0, policy_version 39740 (0.0008) -[2023-10-15 16:23:07,979][52866] Updated weights for policy 1, policy_version 39850 (0.0009) -[2023-10-15 16:23:08,350][52866] Updated weights for policy 1, policy_version 39860 (0.0008) -[2023-10-15 16:23:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 81494016. Throughput: 0: 1772.1, 1: 1820.6. Samples: 20386342. Policy #0 lag: (min: 12.0, avg: 19.3, max: 44.0) -[2023-10-15 16:23:08,442][51532] Avg episode reward: [(0, '36.770'), (1, '44.190')] -[2023-10-15 16:23:08,715][52866] Updated weights for policy 1, policy_version 39870 (0.0009) -[2023-10-15 16:23:08,911][52833] Updated weights for policy 0, policy_version 39750 (0.0008) -[2023-10-15 16:23:09,283][52833] Updated weights for policy 0, policy_version 39760 (0.0008) -[2023-10-15 16:23:09,644][52833] Updated weights for policy 0, policy_version 39770 (0.0008) -[2023-10-15 16:23:12,497][52866] Updated weights for policy 1, policy_version 39880 (0.0008) -[2023-10-15 16:23:12,866][52866] Updated weights for policy 1, policy_version 39890 (0.0008) -[2023-10-15 16:23:13,239][52866] Updated weights for policy 1, policy_version 39900 (0.0009) -[2023-10-15 16:23:13,286][52833] Updated weights for policy 0, policy_version 39780 (0.0008) -[2023-10-15 16:23:13,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 81592320. Throughput: 0: 1791.8, 1: 1813.2. Samples: 20408136. Policy #0 lag: (min: 12.0, avg: 19.3, max: 44.0) -[2023-10-15 16:23:13,441][51532] Avg episode reward: [(0, '38.870'), (1, '43.890')] -[2023-10-15 16:23:13,660][52833] Updated weights for policy 0, policy_version 39790 (0.0010) -[2023-10-15 16:23:14,039][52833] Updated weights for policy 0, policy_version 39800 (0.0010) -[2023-10-15 16:23:17,278][52866] Updated weights for policy 1, policy_version 39910 (0.0008) -[2023-10-15 16:23:17,663][52866] Updated weights for policy 1, policy_version 39920 (0.0011) -[2023-10-15 16:23:17,768][52833] Updated weights for policy 0, policy_version 39810 (0.0009) -[2023-10-15 16:23:18,020][52866] Updated weights for policy 1, policy_version 39930 (0.0007) -[2023-10-15 16:23:18,133][52833] Updated weights for policy 0, policy_version 39820 (0.0008) -[2023-10-15 16:23:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 81657856. Throughput: 0: 1782.6, 1: 1805.0. Samples: 20418832. Policy #0 lag: (min: 12.0, avg: 19.3, max: 44.0) -[2023-10-15 16:23:18,442][51532] Avg episode reward: [(0, '39.200'), (1, '46.800')] -[2023-10-15 16:23:18,498][52833] Updated weights for policy 0, policy_version 39830 (0.0009) -[2023-10-15 16:23:18,865][52833] Updated weights for policy 0, policy_version 39840 (0.0008) -[2023-10-15 16:23:21,695][52866] Updated weights for policy 1, policy_version 39940 (0.0009) -[2023-10-15 16:23:22,057][52866] Updated weights for policy 1, policy_version 39950 (0.0007) -[2023-10-15 16:23:22,420][52866] Updated weights for policy 1, policy_version 39960 (0.0007) -[2023-10-15 16:23:22,526][52833] Updated weights for policy 0, policy_version 39850 (0.0007) -[2023-10-15 16:23:22,895][52833] Updated weights for policy 0, policy_version 39860 (0.0009) -[2023-10-15 16:23:23,262][52833] Updated weights for policy 0, policy_version 39870 (0.0008) -[2023-10-15 16:23:23,441][51532] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 81756160. Throughput: 0: 1793.5, 1: 1812.2. Samples: 20440708. Policy #0 lag: (min: 12.0, avg: 19.3, max: 44.0) -[2023-10-15 16:23:23,441][51532] Avg episode reward: [(0, '39.650'), (1, '44.200')] -[2023-10-15 16:23:26,131][52866] Updated weights for policy 1, policy_version 39970 (0.0009) -[2023-10-15 16:23:26,499][52866] Updated weights for policy 1, policy_version 39980 (0.0008) -[2023-10-15 16:23:26,859][52866] Updated weights for policy 1, policy_version 39990 (0.0008) -[2023-10-15 16:23:27,047][52833] Updated weights for policy 0, policy_version 39880 (0.0007) -[2023-10-15 16:23:27,229][52866] Updated weights for policy 1, policy_version 40000 (0.0007) -[2023-10-15 16:23:27,416][52833] Updated weights for policy 0, policy_version 39890 (0.0008) -[2023-10-15 16:23:27,780][52833] Updated weights for policy 0, policy_version 39900 (0.0010) -[2023-10-15 16:23:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 81821696. Throughput: 0: 1791.6, 1: 1804.2. Samples: 20460886. Policy #0 lag: (min: 12.0, avg: 19.3, max: 44.0) -[2023-10-15 16:23:28,442][51532] Avg episode reward: [(0, '40.110'), (1, '45.460')] -[2023-10-15 16:23:30,991][52866] Updated weights for policy 1, policy_version 40010 (0.0009) -[2023-10-15 16:23:31,358][52866] Updated weights for policy 1, policy_version 40020 (0.0009) -[2023-10-15 16:23:31,507][52833] Updated weights for policy 0, policy_version 39910 (0.0009) -[2023-10-15 16:23:31,732][52866] Updated weights for policy 1, policy_version 40030 (0.0010) -[2023-10-15 16:23:31,875][52833] Updated weights for policy 0, policy_version 39920 (0.0009) -[2023-10-15 16:23:32,249][52833] Updated weights for policy 0, policy_version 39930 (0.0009) -[2023-10-15 16:23:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81887232. Throughput: 0: 1785.4, 1: 1814.8. Samples: 20473046. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:23:33,441][51532] Avg episode reward: [(0, '39.540'), (1, '45.330')] -[2023-10-15 16:23:35,513][52866] Updated weights for policy 1, policy_version 40040 (0.0008) -[2023-10-15 16:23:35,878][52866] Updated weights for policy 1, policy_version 40050 (0.0011) -[2023-10-15 16:23:36,207][52833] Updated weights for policy 0, policy_version 39940 (0.0009) -[2023-10-15 16:23:36,252][52866] Updated weights for policy 1, policy_version 40060 (0.0008) -[2023-10-15 16:23:36,578][52833] Updated weights for policy 0, policy_version 39950 (0.0010) -[2023-10-15 16:23:36,958][52833] Updated weights for policy 0, policy_version 39960 (0.0007) -[2023-10-15 16:23:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81952768. Throughput: 0: 1801.4, 1: 1805.7. Samples: 20493580. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:23:38,442][51532] Avg episode reward: [(0, '39.220'), (1, '45.600')] -[2023-10-15 16:23:40,061][52866] Updated weights for policy 1, policy_version 40070 (0.0010) -[2023-10-15 16:23:40,420][52866] Updated weights for policy 1, policy_version 40080 (0.0010) -[2023-10-15 16:23:40,669][52833] Updated weights for policy 0, policy_version 39970 (0.0007) -[2023-10-15 16:23:40,784][52866] Updated weights for policy 1, policy_version 40090 (0.0008) -[2023-10-15 16:23:41,051][52833] Updated weights for policy 0, policy_version 39980 (0.0008) -[2023-10-15 16:23:41,407][52833] Updated weights for policy 0, policy_version 39990 (0.0009) -[2023-10-15 16:23:41,776][52833] Updated weights for policy 0, policy_version 40000 (0.0008) -[2023-10-15 16:23:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 82018304. Throughput: 0: 1791.1, 1: 1800.3. Samples: 20515412. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:23:43,442][51532] Avg episode reward: [(0, '41.960'), (1, '47.800')] -[2023-10-15 16:23:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000040096_41058304.pth... -[2023-10-15 16:23:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000040000_40960000.pth... -[2023-10-15 16:23:43,482][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000038432_39354368.pth -[2023-10-15 16:23:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000038336_39256064.pth -[2023-10-15 16:23:44,405][52866] Updated weights for policy 1, policy_version 40100 (0.0007) -[2023-10-15 16:23:44,770][52866] Updated weights for policy 1, policy_version 40110 (0.0010) -[2023-10-15 16:23:45,133][52866] Updated weights for policy 1, policy_version 40120 (0.0010) -[2023-10-15 16:23:45,581][52833] Updated weights for policy 0, policy_version 40010 (0.0008) -[2023-10-15 16:23:45,947][52833] Updated weights for policy 0, policy_version 40020 (0.0007) -[2023-10-15 16:23:46,315][52833] Updated weights for policy 0, policy_version 40030 (0.0009) -[2023-10-15 16:23:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 82083840. Throughput: 0: 1806.6, 1: 1800.7. Samples: 20525954. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:23:48,442][51532] Avg episode reward: [(0, '40.640'), (1, '47.010')] -[2023-10-15 16:23:48,909][52866] Updated weights for policy 1, policy_version 40130 (0.0008) -[2023-10-15 16:23:49,276][52866] Updated weights for policy 1, policy_version 40140 (0.0009) -[2023-10-15 16:23:49,643][52866] Updated weights for policy 1, policy_version 40150 (0.0008) -[2023-10-15 16:23:50,004][52866] Updated weights for policy 1, policy_version 40160 (0.0008) -[2023-10-15 16:23:50,143][52833] Updated weights for policy 0, policy_version 40040 (0.0008) -[2023-10-15 16:23:50,516][52833] Updated weights for policy 0, policy_version 40050 (0.0010) -[2023-10-15 16:23:50,877][52833] Updated weights for policy 0, policy_version 40060 (0.0010) -[2023-10-15 16:23:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 82149376. Throughput: 0: 1788.8, 1: 1794.8. Samples: 20547604. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:23:53,441][51532] Avg episode reward: [(0, '39.780'), (1, '46.290')] -[2023-10-15 16:23:53,801][52866] Updated weights for policy 1, policy_version 40170 (0.0009) -[2023-10-15 16:23:54,178][52866] Updated weights for policy 1, policy_version 40180 (0.0009) -[2023-10-15 16:23:54,542][52866] Updated weights for policy 1, policy_version 40190 (0.0008) -[2023-10-15 16:23:54,595][52833] Updated weights for policy 0, policy_version 40070 (0.0009) -[2023-10-15 16:23:54,968][52833] Updated weights for policy 0, policy_version 40080 (0.0010) -[2023-10-15 16:23:55,339][52833] Updated weights for policy 0, policy_version 40090 (0.0009) -[2023-10-15 16:23:58,170][52866] Updated weights for policy 1, policy_version 40200 (0.0010) -[2023-10-15 16:23:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 82214912. Throughput: 0: 1785.7, 1: 1818.7. Samples: 20570334. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:23:58,442][51532] Avg episode reward: [(0, '40.800'), (1, '45.290')] -[2023-10-15 16:23:58,546][52866] Updated weights for policy 1, policy_version 40210 (0.0010) -[2023-10-15 16:23:58,908][52866] Updated weights for policy 1, policy_version 40220 (0.0007) -[2023-10-15 16:23:59,126][52833] Updated weights for policy 0, policy_version 40100 (0.0007) -[2023-10-15 16:23:59,496][52833] Updated weights for policy 0, policy_version 40110 (0.0011) -[2023-10-15 16:23:59,863][52833] Updated weights for policy 0, policy_version 40120 (0.0009) -[2023-10-15 16:24:02,690][52866] Updated weights for policy 1, policy_version 40230 (0.0007) -[2023-10-15 16:24:03,071][52866] Updated weights for policy 1, policy_version 40240 (0.0008) -[2023-10-15 16:24:03,433][52866] Updated weights for policy 1, policy_version 40250 (0.0008) -[2023-10-15 16:24:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 82280448. Throughput: 0: 1779.7, 1: 1804.9. Samples: 20580140. Policy #0 lag: (min: 23.0, avg: 30.1, max: 55.0) -[2023-10-15 16:24:03,442][51532] Avg episode reward: [(0, '41.420'), (1, '45.590')] -[2023-10-15 16:24:03,668][52833] Updated weights for policy 0, policy_version 40130 (0.0009) -[2023-10-15 16:24:04,030][52833] Updated weights for policy 0, policy_version 40140 (0.0008) -[2023-10-15 16:24:04,386][52833] Updated weights for policy 0, policy_version 40150 (0.0009) -[2023-10-15 16:24:04,756][52833] Updated weights for policy 0, policy_version 40160 (0.0008) -[2023-10-15 16:24:07,112][52866] Updated weights for policy 1, policy_version 40260 (0.0009) -[2023-10-15 16:24:07,473][52866] Updated weights for policy 1, policy_version 40270 (0.0008) -[2023-10-15 16:24:07,839][52866] Updated weights for policy 1, policy_version 40280 (0.0008) -[2023-10-15 16:24:08,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 82378752. Throughput: 0: 1776.3, 1: 1815.5. Samples: 20602340. Policy #0 lag: (min: 23.0, avg: 30.1, max: 55.0) -[2023-10-15 16:24:08,442][51532] Avg episode reward: [(0, '42.020'), (1, '49.890')] -[2023-10-15 16:24:08,443][52518] Saving new best policy, reward=49.890! -[2023-10-15 16:24:08,595][52833] Updated weights for policy 0, policy_version 40170 (0.0007) -[2023-10-15 16:24:08,958][52833] Updated weights for policy 0, policy_version 40180 (0.0007) -[2023-10-15 16:24:09,333][52833] Updated weights for policy 0, policy_version 40190 (0.0010) -[2023-10-15 16:24:11,606][52866] Updated weights for policy 1, policy_version 40290 (0.0008) -[2023-10-15 16:24:11,973][52866] Updated weights for policy 1, policy_version 40300 (0.0010) -[2023-10-15 16:24:12,343][52866] Updated weights for policy 1, policy_version 40310 (0.0010) -[2023-10-15 16:24:12,706][52866] Updated weights for policy 1, policy_version 40320 (0.0008) -[2023-10-15 16:24:13,221][52833] Updated weights for policy 0, policy_version 40200 (0.0008) -[2023-10-15 16:24:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 82444288. Throughput: 0: 1814.0, 1: 1806.0. Samples: 20623784. Policy #0 lag: (min: 23.0, avg: 30.1, max: 55.0) -[2023-10-15 16:24:13,442][51532] Avg episode reward: [(0, '41.970'), (1, '48.490')] -[2023-10-15 16:24:13,589][52833] Updated weights for policy 0, policy_version 40210 (0.0007) -[2023-10-15 16:24:13,943][52833] Updated weights for policy 0, policy_version 40220 (0.0007) -[2023-10-15 16:24:16,481][52866] Updated weights for policy 1, policy_version 40330 (0.0009) -[2023-10-15 16:24:16,854][52866] Updated weights for policy 1, policy_version 40340 (0.0008) -[2023-10-15 16:24:17,220][52866] Updated weights for policy 1, policy_version 40350 (0.0008) -[2023-10-15 16:24:17,741][52833] Updated weights for policy 0, policy_version 40230 (0.0007) -[2023-10-15 16:24:18,111][52833] Updated weights for policy 0, policy_version 40240 (0.0008) -[2023-10-15 16:24:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 82509824. Throughput: 0: 1784.5, 1: 1816.2. Samples: 20635078. Policy #0 lag: (min: 23.0, avg: 30.1, max: 55.0) -[2023-10-15 16:24:18,442][51532] Avg episode reward: [(0, '42.090'), (1, '46.460')] -[2023-10-15 16:24:18,499][52833] Updated weights for policy 0, policy_version 40250 (0.0010) -[2023-10-15 16:24:20,856][52866] Updated weights for policy 1, policy_version 40360 (0.0007) -[2023-10-15 16:24:21,229][52866] Updated weights for policy 1, policy_version 40370 (0.0008) -[2023-10-15 16:24:21,600][52866] Updated weights for policy 1, policy_version 40380 (0.0011) -[2023-10-15 16:24:22,311][52833] Updated weights for policy 0, policy_version 40260 (0.0010) -[2023-10-15 16:24:22,680][52833] Updated weights for policy 0, policy_version 40270 (0.0007) -[2023-10-15 16:24:23,053][52833] Updated weights for policy 0, policy_version 40280 (0.0009) -[2023-10-15 16:24:23,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 82608128. Throughput: 0: 1800.9, 1: 1808.7. Samples: 20656012. Policy #0 lag: (min: 23.0, avg: 30.1, max: 55.0) -[2023-10-15 16:24:23,441][51532] Avg episode reward: [(0, '44.410'), (1, '45.740')] -[2023-10-15 16:24:25,222][52866] Updated weights for policy 1, policy_version 40390 (0.0009) -[2023-10-15 16:24:25,581][52866] Updated weights for policy 1, policy_version 40400 (0.0007) -[2023-10-15 16:24:25,949][52866] Updated weights for policy 1, policy_version 40410 (0.0007) -[2023-10-15 16:24:26,822][52833] Updated weights for policy 0, policy_version 40290 (0.0009) -[2023-10-15 16:24:27,189][52833] Updated weights for policy 0, policy_version 40300 (0.0009) -[2023-10-15 16:24:27,553][52833] Updated weights for policy 0, policy_version 40310 (0.0008) -[2023-10-15 16:24:27,927][52833] Updated weights for policy 0, policy_version 40320 (0.0007) -[2023-10-15 16:24:28,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 82673664. Throughput: 0: 1783.0, 1: 1815.6. Samples: 20677350. Policy #0 lag: (min: 10.0, avg: 30.9, max: 32.0) -[2023-10-15 16:24:28,442][51532] Avg episode reward: [(0, '40.730'), (1, '45.880')] -[2023-10-15 16:24:29,716][52866] Updated weights for policy 1, policy_version 40420 (0.0008) -[2023-10-15 16:24:30,088][52866] Updated weights for policy 1, policy_version 40430 (0.0007) -[2023-10-15 16:24:30,453][52866] Updated weights for policy 1, policy_version 40440 (0.0007) -[2023-10-15 16:24:31,557][52833] Updated weights for policy 0, policy_version 40330 (0.0009) -[2023-10-15 16:24:31,923][52833] Updated weights for policy 0, policy_version 40340 (0.0007) -[2023-10-15 16:24:32,285][52833] Updated weights for policy 0, policy_version 40350 (0.0008) -[2023-10-15 16:24:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 82739200. Throughput: 0: 1797.9, 1: 1813.9. Samples: 20688486. Policy #0 lag: (min: 10.0, avg: 30.9, max: 32.0) -[2023-10-15 16:24:33,442][51532] Avg episode reward: [(0, '42.650'), (1, '42.330')] -[2023-10-15 16:24:34,016][52866] Updated weights for policy 1, policy_version 40450 (0.0007) -[2023-10-15 16:24:34,397][52866] Updated weights for policy 1, policy_version 40460 (0.0009) -[2023-10-15 16:24:34,754][52866] Updated weights for policy 1, policy_version 40470 (0.0009) -[2023-10-15 16:24:35,126][52866] Updated weights for policy 1, policy_version 40480 (0.0008) -[2023-10-15 16:24:35,956][52833] Updated weights for policy 0, policy_version 40360 (0.0008) -[2023-10-15 16:24:36,323][52833] Updated weights for policy 0, policy_version 40370 (0.0008) -[2023-10-15 16:24:36,697][52833] Updated weights for policy 0, policy_version 40380 (0.0010) -[2023-10-15 16:24:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 82804736. Throughput: 0: 1788.3, 1: 1810.2. Samples: 20709536. Policy #0 lag: (min: 10.0, avg: 30.9, max: 32.0) -[2023-10-15 16:24:38,442][51532] Avg episode reward: [(0, '39.860'), (1, '42.350')] -[2023-10-15 16:24:38,970][52866] Updated weights for policy 1, policy_version 40490 (0.0009) -[2023-10-15 16:24:39,337][52866] Updated weights for policy 1, policy_version 40500 (0.0009) -[2023-10-15 16:24:39,697][52866] Updated weights for policy 1, policy_version 40510 (0.0007) -[2023-10-15 16:24:40,209][52833] Updated weights for policy 0, policy_version 40390 (0.0011) -[2023-10-15 16:24:40,582][52833] Updated weights for policy 0, policy_version 40400 (0.0011) -[2023-10-15 16:24:40,951][52833] Updated weights for policy 0, policy_version 40410 (0.0010) -[2023-10-15 16:24:43,315][52866] Updated weights for policy 1, policy_version 40520 (0.0007) -[2023-10-15 16:24:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 82870272. Throughput: 0: 1784.1, 1: 1811.7. Samples: 20732146. Policy #0 lag: (min: 10.0, avg: 30.9, max: 32.0) -[2023-10-15 16:24:43,441][51532] Avg episode reward: [(0, '41.510'), (1, '44.220')] -[2023-10-15 16:24:43,689][52866] Updated weights for policy 1, policy_version 40530 (0.0009) -[2023-10-15 16:24:44,064][52866] Updated weights for policy 1, policy_version 40540 (0.0008) -[2023-10-15 16:24:44,788][52833] Updated weights for policy 0, policy_version 40420 (0.0009) -[2023-10-15 16:24:45,165][52833] Updated weights for policy 0, policy_version 40430 (0.0010) -[2023-10-15 16:24:45,536][52833] Updated weights for policy 0, policy_version 40440 (0.0008) -[2023-10-15 16:24:48,056][52866] Updated weights for policy 1, policy_version 40550 (0.0008) -[2023-10-15 16:24:48,437][52866] Updated weights for policy 1, policy_version 40560 (0.0011) -[2023-10-15 16:24:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 82935808. Throughput: 0: 1787.8, 1: 1806.4. Samples: 20741880. Policy #0 lag: (min: 10.0, avg: 30.9, max: 32.0) -[2023-10-15 16:24:48,441][51532] Avg episode reward: [(0, '39.440'), (1, '43.430')] -[2023-10-15 16:24:48,807][52866] Updated weights for policy 1, policy_version 40570 (0.0009) -[2023-10-15 16:24:49,396][52833] Updated weights for policy 0, policy_version 40450 (0.0007) -[2023-10-15 16:24:49,767][52833] Updated weights for policy 0, policy_version 40460 (0.0008) -[2023-10-15 16:24:50,132][52833] Updated weights for policy 0, policy_version 40470 (0.0007) -[2023-10-15 16:24:50,505][52833] Updated weights for policy 0, policy_version 40480 (0.0009) -[2023-10-15 16:24:52,506][52866] Updated weights for policy 1, policy_version 40580 (0.0008) -[2023-10-15 16:24:52,877][52866] Updated weights for policy 1, policy_version 40590 (0.0007) -[2023-10-15 16:24:53,242][52866] Updated weights for policy 1, policy_version 40600 (0.0008) -[2023-10-15 16:24:53,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 83001344. Throughput: 0: 1793.0, 1: 1805.6. Samples: 20764274. Policy #0 lag: (min: 10.0, avg: 30.9, max: 32.0) -[2023-10-15 16:24:53,442][51532] Avg episode reward: [(0, '39.990'), (1, '41.720')] -[2023-10-15 16:24:54,219][52833] Updated weights for policy 0, policy_version 40490 (0.0010) -[2023-10-15 16:24:54,599][52833] Updated weights for policy 0, policy_version 40500 (0.0011) -[2023-10-15 16:24:54,965][52833] Updated weights for policy 0, policy_version 40510 (0.0009) -[2023-10-15 16:24:56,841][52866] Updated weights for policy 1, policy_version 40610 (0.0009) -[2023-10-15 16:24:57,218][52866] Updated weights for policy 1, policy_version 40620 (0.0008) -[2023-10-15 16:24:57,586][52866] Updated weights for policy 1, policy_version 40630 (0.0007) -[2023-10-15 16:24:57,942][52866] Updated weights for policy 1, policy_version 40640 (0.0008) -[2023-10-15 16:24:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 83099648. Throughput: 0: 1788.4, 1: 1803.8. Samples: 20785432. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 16:24:58,441][51532] Avg episode reward: [(0, '39.680'), (1, '42.460')] -[2023-10-15 16:24:58,796][52833] Updated weights for policy 0, policy_version 40520 (0.0009) -[2023-10-15 16:24:59,162][52833] Updated weights for policy 0, policy_version 40530 (0.0008) -[2023-10-15 16:24:59,539][52833] Updated weights for policy 0, policy_version 40540 (0.0010) -[2023-10-15 16:25:01,400][52866] Updated weights for policy 1, policy_version 40650 (0.0008) -[2023-10-15 16:25:01,766][52866] Updated weights for policy 1, policy_version 40660 (0.0008) -[2023-10-15 16:25:02,134][52866] Updated weights for policy 1, policy_version 40670 (0.0008) -[2023-10-15 16:25:03,232][52833] Updated weights for policy 0, policy_version 40550 (0.0008) -[2023-10-15 16:25:03,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 83165184. Throughput: 0: 1784.7, 1: 1810.6. Samples: 20796866. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 16:25:03,441][51532] Avg episode reward: [(0, '38.950'), (1, '42.600')] -[2023-10-15 16:25:03,600][52833] Updated weights for policy 0, policy_version 40560 (0.0007) -[2023-10-15 16:25:03,958][52833] Updated weights for policy 0, policy_version 40570 (0.0009) -[2023-10-15 16:25:05,955][52866] Updated weights for policy 1, policy_version 40680 (0.0008) -[2023-10-15 16:25:06,316][52866] Updated weights for policy 1, policy_version 40690 (0.0009) -[2023-10-15 16:25:06,678][52866] Updated weights for policy 1, policy_version 40700 (0.0008) -[2023-10-15 16:25:07,711][52833] Updated weights for policy 0, policy_version 40580 (0.0010) -[2023-10-15 16:25:08,081][52833] Updated weights for policy 0, policy_version 40590 (0.0007) -[2023-10-15 16:25:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 83230720. Throughput: 0: 1791.7, 1: 1811.6. Samples: 20818160. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 16:25:08,441][51532] Avg episode reward: [(0, '38.750'), (1, '42.340')] -[2023-10-15 16:25:08,445][52833] Updated weights for policy 0, policy_version 40600 (0.0008) -[2023-10-15 16:25:10,413][52866] Updated weights for policy 1, policy_version 40710 (0.0008) -[2023-10-15 16:25:10,784][52866] Updated weights for policy 1, policy_version 40720 (0.0010) -[2023-10-15 16:25:11,157][52866] Updated weights for policy 1, policy_version 40730 (0.0010) -[2023-10-15 16:25:12,289][52833] Updated weights for policy 0, policy_version 40610 (0.0008) -[2023-10-15 16:25:12,665][52833] Updated weights for policy 0, policy_version 40620 (0.0008) -[2023-10-15 16:25:13,034][52833] Updated weights for policy 0, policy_version 40630 (0.0011) -[2023-10-15 16:25:13,403][52833] Updated weights for policy 0, policy_version 40640 (0.0010) -[2023-10-15 16:25:13,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 83329024. Throughput: 0: 1802.9, 1: 1809.4. Samples: 20839904. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 16:25:13,442][51532] Avg episode reward: [(0, '37.950'), (1, '40.400')] -[2023-10-15 16:25:14,792][52866] Updated weights for policy 1, policy_version 40740 (0.0010) -[2023-10-15 16:25:15,167][52866] Updated weights for policy 1, policy_version 40750 (0.0009) -[2023-10-15 16:25:15,531][52866] Updated weights for policy 1, policy_version 40760 (0.0007) -[2023-10-15 16:25:17,177][52833] Updated weights for policy 0, policy_version 40650 (0.0008) -[2023-10-15 16:25:17,541][52833] Updated weights for policy 0, policy_version 40660 (0.0007) -[2023-10-15 16:25:17,920][52833] Updated weights for policy 0, policy_version 40670 (0.0008) -[2023-10-15 16:25:18,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 83394560. Throughput: 0: 1788.1, 1: 1807.7. Samples: 20850298. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-15 16:25:18,442][51532] Avg episode reward: [(0, '38.880'), (1, '42.210')] -[2023-10-15 16:25:19,318][52866] Updated weights for policy 1, policy_version 40770 (0.0009) -[2023-10-15 16:25:19,684][52866] Updated weights for policy 1, policy_version 40780 (0.0008) -[2023-10-15 16:25:20,054][52866] Updated weights for policy 1, policy_version 40790 (0.0009) -[2023-10-15 16:25:20,417][52866] Updated weights for policy 1, policy_version 40800 (0.0008) -[2023-10-15 16:25:21,650][52833] Updated weights for policy 0, policy_version 40680 (0.0010) -[2023-10-15 16:25:22,015][52833] Updated weights for policy 0, policy_version 40690 (0.0010) -[2023-10-15 16:25:22,393][52833] Updated weights for policy 0, policy_version 40700 (0.0010) -[2023-10-15 16:25:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 83460096. Throughput: 0: 1801.0, 1: 1809.9. Samples: 20872026. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:25:23,442][51532] Avg episode reward: [(0, '39.910'), (1, '44.780')] -[2023-10-15 16:25:24,016][52866] Updated weights for policy 1, policy_version 40810 (0.0008) -[2023-10-15 16:25:24,380][52866] Updated weights for policy 1, policy_version 40820 (0.0009) -[2023-10-15 16:25:24,750][52866] Updated weights for policy 1, policy_version 40830 (0.0009) -[2023-10-15 16:25:26,204][52833] Updated weights for policy 0, policy_version 40710 (0.0008) -[2023-10-15 16:25:26,581][52833] Updated weights for policy 0, policy_version 40720 (0.0008) -[2023-10-15 16:25:26,952][52833] Updated weights for policy 0, policy_version 40730 (0.0009) -[2023-10-15 16:25:28,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83525632. Throughput: 0: 1782.2, 1: 1810.3. Samples: 20893806. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:25:28,441][51532] Avg episode reward: [(0, '40.170'), (1, '44.990')] -[2023-10-15 16:25:28,515][52866] Updated weights for policy 1, policy_version 40840 (0.0008) -[2023-10-15 16:25:28,886][52866] Updated weights for policy 1, policy_version 40850 (0.0008) -[2023-10-15 16:25:29,251][52866] Updated weights for policy 1, policy_version 40860 (0.0007) -[2023-10-15 16:25:30,711][52833] Updated weights for policy 0, policy_version 40740 (0.0009) -[2023-10-15 16:25:31,088][52833] Updated weights for policy 0, policy_version 40750 (0.0007) -[2023-10-15 16:25:31,461][52833] Updated weights for policy 0, policy_version 40760 (0.0008) -[2023-10-15 16:25:32,972][52866] Updated weights for policy 1, policy_version 40870 (0.0007) -[2023-10-15 16:25:33,343][52866] Updated weights for policy 1, policy_version 40880 (0.0009) -[2023-10-15 16:25:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 83591168. Throughput: 0: 1807.2, 1: 1813.9. Samples: 20904832. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:25:33,441][51532] Avg episode reward: [(0, '40.420'), (1, '44.030')] -[2023-10-15 16:25:33,715][52866] Updated weights for policy 1, policy_version 40890 (0.0009) -[2023-10-15 16:25:35,110][52833] Updated weights for policy 0, policy_version 40770 (0.0008) -[2023-10-15 16:25:35,475][52833] Updated weights for policy 0, policy_version 40780 (0.0008) -[2023-10-15 16:25:35,851][52833] Updated weights for policy 0, policy_version 40790 (0.0008) -[2023-10-15 16:25:36,215][52833] Updated weights for policy 0, policy_version 40800 (0.0009) -[2023-10-15 16:25:37,482][52866] Updated weights for policy 1, policy_version 40900 (0.0008) -[2023-10-15 16:25:37,853][52866] Updated weights for policy 1, policy_version 40910 (0.0008) -[2023-10-15 16:25:38,220][52866] Updated weights for policy 1, policy_version 40920 (0.0008) -[2023-10-15 16:25:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 83656704. Throughput: 0: 1779.7, 1: 1817.7. Samples: 20926152. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:25:38,441][51532] Avg episode reward: [(0, '40.250'), (1, '42.520')] -[2023-10-15 16:25:40,016][52833] Updated weights for policy 0, policy_version 40810 (0.0009) -[2023-10-15 16:25:40,397][52833] Updated weights for policy 0, policy_version 40820 (0.0010) -[2023-10-15 16:25:40,773][52833] Updated weights for policy 0, policy_version 40830 (0.0007) -[2023-10-15 16:25:41,843][52866] Updated weights for policy 1, policy_version 40930 (0.0008) -[2023-10-15 16:25:42,205][52866] Updated weights for policy 1, policy_version 40940 (0.0011) -[2023-10-15 16:25:42,575][52866] Updated weights for policy 1, policy_version 40950 (0.0009) -[2023-10-15 16:25:42,944][52866] Updated weights for policy 1, policy_version 40960 (0.0008) -[2023-10-15 16:25:43,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 83755008. Throughput: 0: 1778.4, 1: 1819.1. Samples: 20947318. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:25:43,441][51532] Avg episode reward: [(0, '42.690'), (1, '41.230')] -[2023-10-15 16:25:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000040832_41811968.pth... -[2023-10-15 16:25:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000040960_41943040.pth... -[2023-10-15 16:25:43,490][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000039168_40108032.pth -[2023-10-15 16:25:43,490][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000039264_40206336.pth -[2023-10-15 16:25:44,614][52833] Updated weights for policy 0, policy_version 40840 (0.0010) -[2023-10-15 16:25:44,983][52833] Updated weights for policy 0, policy_version 40850 (0.0008) -[2023-10-15 16:25:45,353][52833] Updated weights for policy 0, policy_version 40860 (0.0010) -[2023-10-15 16:25:46,746][52866] Updated weights for policy 1, policy_version 40970 (0.0008) -[2023-10-15 16:25:47,102][52866] Updated weights for policy 1, policy_version 40980 (0.0008) -[2023-10-15 16:25:47,478][52866] Updated weights for policy 1, policy_version 40990 (0.0009) -[2023-10-15 16:25:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 83820544. Throughput: 0: 1779.4, 1: 1809.5. Samples: 20958364. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:25:48,441][51532] Avg episode reward: [(0, '41.040'), (1, '42.880')] -[2023-10-15 16:25:49,044][52833] Updated weights for policy 0, policy_version 40870 (0.0007) -[2023-10-15 16:25:49,403][52833] Updated weights for policy 0, policy_version 40880 (0.0007) -[2023-10-15 16:25:49,775][52833] Updated weights for policy 0, policy_version 40890 (0.0009) -[2023-10-15 16:25:51,132][52866] Updated weights for policy 1, policy_version 41000 (0.0008) -[2023-10-15 16:25:51,495][52866] Updated weights for policy 1, policy_version 41010 (0.0010) -[2023-10-15 16:25:51,872][52866] Updated weights for policy 1, policy_version 41020 (0.0011) -[2023-10-15 16:25:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 83886080. Throughput: 0: 1779.7, 1: 1806.3. Samples: 20979528. Policy #0 lag: (min: 1.0, avg: 15.0, max: 33.0) -[2023-10-15 16:25:53,441][51532] Avg episode reward: [(0, '41.590'), (1, '40.680')] -[2023-10-15 16:25:53,517][52833] Updated weights for policy 0, policy_version 40900 (0.0007) -[2023-10-15 16:25:53,896][52833] Updated weights for policy 0, policy_version 40910 (0.0007) -[2023-10-15 16:25:54,264][52833] Updated weights for policy 0, policy_version 40920 (0.0007) -[2023-10-15 16:25:55,715][52866] Updated weights for policy 1, policy_version 41030 (0.0011) -[2023-10-15 16:25:56,080][52866] Updated weights for policy 1, policy_version 41040 (0.0009) -[2023-10-15 16:25:56,452][52866] Updated weights for policy 1, policy_version 41050 (0.0008) -[2023-10-15 16:25:58,062][52833] Updated weights for policy 0, policy_version 40930 (0.0008) -[2023-10-15 16:25:58,429][52833] Updated weights for policy 0, policy_version 40940 (0.0008) -[2023-10-15 16:25:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 83951616. Throughput: 0: 1800.4, 1: 1794.5. Samples: 21001670. Policy #0 lag: (min: 1.0, avg: 15.0, max: 33.0) -[2023-10-15 16:25:58,441][51532] Avg episode reward: [(0, '39.650'), (1, '39.850')] -[2023-10-15 16:25:58,791][52833] Updated weights for policy 0, policy_version 40950 (0.0008) -[2023-10-15 16:25:59,162][52833] Updated weights for policy 0, policy_version 40960 (0.0007) -[2023-10-15 16:26:00,251][52866] Updated weights for policy 1, policy_version 41060 (0.0009) -[2023-10-15 16:26:00,612][52866] Updated weights for policy 1, policy_version 41070 (0.0008) -[2023-10-15 16:26:00,975][52866] Updated weights for policy 1, policy_version 41080 (0.0007) -[2023-10-15 16:26:02,835][52833] Updated weights for policy 0, policy_version 40970 (0.0010) -[2023-10-15 16:26:03,212][52833] Updated weights for policy 0, policy_version 40980 (0.0007) -[2023-10-15 16:26:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 84017152. Throughput: 0: 1786.9, 1: 1805.1. Samples: 21011940. Policy #0 lag: (min: 1.0, avg: 15.0, max: 33.0) -[2023-10-15 16:26:03,442][51532] Avg episode reward: [(0, '37.900'), (1, '38.860')] -[2023-10-15 16:26:03,573][52833] Updated weights for policy 0, policy_version 40990 (0.0007) -[2023-10-15 16:26:04,680][52866] Updated weights for policy 1, policy_version 41090 (0.0009) -[2023-10-15 16:26:05,046][52866] Updated weights for policy 1, policy_version 41100 (0.0010) -[2023-10-15 16:26:05,422][52866] Updated weights for policy 1, policy_version 41110 (0.0009) -[2023-10-15 16:26:05,793][52866] Updated weights for policy 1, policy_version 41120 (0.0011) -[2023-10-15 16:26:07,408][52833] Updated weights for policy 0, policy_version 41000 (0.0008) -[2023-10-15 16:26:07,780][52833] Updated weights for policy 0, policy_version 41010 (0.0009) -[2023-10-15 16:26:08,149][52833] Updated weights for policy 0, policy_version 41020 (0.0007) -[2023-10-15 16:26:08,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 84115456. Throughput: 0: 1798.8, 1: 1794.0. Samples: 21033702. Policy #0 lag: (min: 1.0, avg: 15.0, max: 33.0) -[2023-10-15 16:26:08,442][51532] Avg episode reward: [(0, '39.730'), (1, '35.960')] -[2023-10-15 16:26:09,554][52866] Updated weights for policy 1, policy_version 41130 (0.0010) -[2023-10-15 16:26:09,923][52866] Updated weights for policy 1, policy_version 41140 (0.0009) -[2023-10-15 16:26:10,288][52866] Updated weights for policy 1, policy_version 41150 (0.0009) -[2023-10-15 16:26:11,935][52833] Updated weights for policy 0, policy_version 41030 (0.0008) -[2023-10-15 16:26:12,301][52833] Updated weights for policy 0, policy_version 41040 (0.0008) -[2023-10-15 16:26:12,677][52833] Updated weights for policy 0, policy_version 41050 (0.0009) -[2023-10-15 16:26:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 84180992. Throughput: 0: 1783.1, 1: 1790.5. Samples: 21054616. Policy #0 lag: (min: 1.0, avg: 15.0, max: 33.0) -[2023-10-15 16:26:13,442][51532] Avg episode reward: [(0, '39.800'), (1, '36.830')] -[2023-10-15 16:26:14,051][52866] Updated weights for policy 1, policy_version 41160 (0.0011) -[2023-10-15 16:26:14,423][52866] Updated weights for policy 1, policy_version 41170 (0.0009) -[2023-10-15 16:26:14,776][52866] Updated weights for policy 1, policy_version 41180 (0.0009) -[2023-10-15 16:26:16,487][52833] Updated weights for policy 0, policy_version 41060 (0.0008) -[2023-10-15 16:26:16,868][52833] Updated weights for policy 0, policy_version 41070 (0.0007) -[2023-10-15 16:26:17,243][52833] Updated weights for policy 0, policy_version 41080 (0.0009) -[2023-10-15 16:26:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 84246528. Throughput: 0: 1785.8, 1: 1790.3. Samples: 21065754. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:26:18,442][51532] Avg episode reward: [(0, '40.050'), (1, '37.110')] -[2023-10-15 16:26:18,528][52866] Updated weights for policy 1, policy_version 41190 (0.0010) -[2023-10-15 16:26:18,903][52866] Updated weights for policy 1, policy_version 41200 (0.0008) -[2023-10-15 16:26:19,271][52866] Updated weights for policy 1, policy_version 41210 (0.0009) -[2023-10-15 16:26:20,958][52833] Updated weights for policy 0, policy_version 41090 (0.0010) -[2023-10-15 16:26:21,330][52833] Updated weights for policy 0, policy_version 41100 (0.0008) -[2023-10-15 16:26:21,691][52833] Updated weights for policy 0, policy_version 41110 (0.0008) -[2023-10-15 16:26:22,063][52833] Updated weights for policy 0, policy_version 41120 (0.0007) -[2023-10-15 16:26:22,814][52866] Updated weights for policy 1, policy_version 41220 (0.0009) -[2023-10-15 16:26:23,181][52866] Updated weights for policy 1, policy_version 41230 (0.0008) -[2023-10-15 16:26:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 84312064. Throughput: 0: 1786.9, 1: 1798.2. Samples: 21087480. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:26:23,442][51532] Avg episode reward: [(0, '40.220'), (1, '37.560')] -[2023-10-15 16:26:23,559][52866] Updated weights for policy 1, policy_version 41240 (0.0007) -[2023-10-15 16:26:25,861][52833] Updated weights for policy 0, policy_version 41130 (0.0007) -[2023-10-15 16:26:26,227][52833] Updated weights for policy 0, policy_version 41140 (0.0009) -[2023-10-15 16:26:26,595][52833] Updated weights for policy 0, policy_version 41150 (0.0009) -[2023-10-15 16:26:27,335][52866] Updated weights for policy 1, policy_version 41250 (0.0009) -[2023-10-15 16:26:27,702][52866] Updated weights for policy 1, policy_version 41260 (0.0008) -[2023-10-15 16:26:28,071][52866] Updated weights for policy 1, policy_version 41270 (0.0009) -[2023-10-15 16:26:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 84377600. Throughput: 0: 1780.0, 1: 1809.4. Samples: 21108840. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:26:28,442][51532] Avg episode reward: [(0, '41.300'), (1, '40.650')] -[2023-10-15 16:26:28,447][52866] Updated weights for policy 1, policy_version 41280 (0.0010) -[2023-10-15 16:26:30,500][52833] Updated weights for policy 0, policy_version 41160 (0.0009) -[2023-10-15 16:26:30,882][52833] Updated weights for policy 0, policy_version 41170 (0.0008) -[2023-10-15 16:26:31,242][52833] Updated weights for policy 0, policy_version 41180 (0.0009) -[2023-10-15 16:26:32,183][52866] Updated weights for policy 1, policy_version 41290 (0.0008) -[2023-10-15 16:26:32,546][52866] Updated weights for policy 1, policy_version 41300 (0.0007) -[2023-10-15 16:26:32,923][52866] Updated weights for policy 1, policy_version 41310 (0.0009) -[2023-10-15 16:26:33,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 84475904. Throughput: 0: 1796.8, 1: 1797.9. Samples: 21120122. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:26:33,441][51532] Avg episode reward: [(0, '38.230'), (1, '39.380')] -[2023-10-15 16:26:34,913][52833] Updated weights for policy 0, policy_version 41190 (0.0009) -[2023-10-15 16:26:35,284][52833] Updated weights for policy 0, policy_version 41200 (0.0008) -[2023-10-15 16:26:35,653][52833] Updated weights for policy 0, policy_version 41210 (0.0008) -[2023-10-15 16:26:36,679][52866] Updated weights for policy 1, policy_version 41320 (0.0009) -[2023-10-15 16:26:37,039][52866] Updated weights for policy 1, policy_version 41330 (0.0008) -[2023-10-15 16:26:37,406][52866] Updated weights for policy 1, policy_version 41340 (0.0008) -[2023-10-15 16:26:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 84541440. Throughput: 0: 1780.3, 1: 1815.3. Samples: 21141334. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:26:38,442][51532] Avg episode reward: [(0, '41.250'), (1, '39.650')] -[2023-10-15 16:26:39,494][52833] Updated weights for policy 0, policy_version 41220 (0.0009) -[2023-10-15 16:26:39,869][52833] Updated weights for policy 0, policy_version 41230 (0.0008) -[2023-10-15 16:26:40,241][52833] Updated weights for policy 0, policy_version 41240 (0.0008) -[2023-10-15 16:26:41,177][52866] Updated weights for policy 1, policy_version 41350 (0.0008) -[2023-10-15 16:26:41,537][52866] Updated weights for policy 1, policy_version 41360 (0.0008) -[2023-10-15 16:26:41,917][52866] Updated weights for policy 1, policy_version 41370 (0.0010) -[2023-10-15 16:26:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 84606976. Throughput: 0: 1780.6, 1: 1805.8. Samples: 21163058. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:26:43,442][51532] Avg episode reward: [(0, '42.680'), (1, '40.820')] -[2023-10-15 16:26:44,062][52833] Updated weights for policy 0, policy_version 41250 (0.0008) -[2023-10-15 16:26:44,441][52833] Updated weights for policy 0, policy_version 41260 (0.0007) -[2023-10-15 16:26:44,809][52833] Updated weights for policy 0, policy_version 41270 (0.0009) -[2023-10-15 16:26:45,168][52833] Updated weights for policy 0, policy_version 41280 (0.0008) -[2023-10-15 16:26:45,516][52866] Updated weights for policy 1, policy_version 41380 (0.0008) -[2023-10-15 16:26:45,878][52866] Updated weights for policy 1, policy_version 41390 (0.0008) -[2023-10-15 16:26:46,242][52866] Updated weights for policy 1, policy_version 41400 (0.0007) -[2023-10-15 16:26:48,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 84672512. Throughput: 0: 1781.1, 1: 1813.6. Samples: 21173700. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) -[2023-10-15 16:26:48,441][51532] Avg episode reward: [(0, '41.660'), (1, '43.800')] -[2023-10-15 16:26:48,995][52833] Updated weights for policy 0, policy_version 41290 (0.0010) -[2023-10-15 16:26:49,364][52833] Updated weights for policy 0, policy_version 41300 (0.0007) -[2023-10-15 16:26:49,740][52833] Updated weights for policy 0, policy_version 41310 (0.0010) -[2023-10-15 16:26:49,991][52866] Updated weights for policy 1, policy_version 41410 (0.0009) -[2023-10-15 16:26:50,360][52866] Updated weights for policy 1, policy_version 41420 (0.0007) -[2023-10-15 16:26:50,721][52866] Updated weights for policy 1, policy_version 41430 (0.0007) -[2023-10-15 16:26:51,095][52866] Updated weights for policy 1, policy_version 41440 (0.0007) -[2023-10-15 16:26:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 84738048. Throughput: 0: 1781.9, 1: 1804.1. Samples: 21195070. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) -[2023-10-15 16:26:53,441][51532] Avg episode reward: [(0, '40.620'), (1, '41.860')] -[2023-10-15 16:26:53,469][52833] Updated weights for policy 0, policy_version 41320 (0.0009) -[2023-10-15 16:26:53,848][52833] Updated weights for policy 0, policy_version 41330 (0.0009) -[2023-10-15 16:26:54,211][52833] Updated weights for policy 0, policy_version 41340 (0.0008) -[2023-10-15 16:26:54,897][52866] Updated weights for policy 1, policy_version 41450 (0.0008) -[2023-10-15 16:26:55,255][52866] Updated weights for policy 1, policy_version 41460 (0.0011) -[2023-10-15 16:26:55,624][52866] Updated weights for policy 1, policy_version 41470 (0.0007) -[2023-10-15 16:26:57,935][52833] Updated weights for policy 0, policy_version 41350 (0.0009) -[2023-10-15 16:26:58,308][52833] Updated weights for policy 0, policy_version 41360 (0.0010) -[2023-10-15 16:26:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 84803584. Throughput: 0: 1808.9, 1: 1803.0. Samples: 21217150. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) -[2023-10-15 16:26:58,441][51532] Avg episode reward: [(0, '40.970'), (1, '43.030')] -[2023-10-15 16:26:58,675][52833] Updated weights for policy 0, policy_version 41370 (0.0008) -[2023-10-15 16:26:59,476][52866] Updated weights for policy 1, policy_version 41480 (0.0008) -[2023-10-15 16:26:59,850][52866] Updated weights for policy 1, policy_version 41490 (0.0009) -[2023-10-15 16:27:00,221][52866] Updated weights for policy 1, policy_version 41500 (0.0007) -[2023-10-15 16:27:02,342][52833] Updated weights for policy 0, policy_version 41380 (0.0008) -[2023-10-15 16:27:02,717][52833] Updated weights for policy 0, policy_version 41390 (0.0007) -[2023-10-15 16:27:03,079][52833] Updated weights for policy 0, policy_version 41400 (0.0008) -[2023-10-15 16:27:03,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 84901888. Throughput: 0: 1784.5, 1: 1803.0. Samples: 21227194. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) -[2023-10-15 16:27:03,441][51532] Avg episode reward: [(0, '43.850'), (1, '42.730')] -[2023-10-15 16:27:04,041][52866] Updated weights for policy 1, policy_version 41510 (0.0008) -[2023-10-15 16:27:04,408][52866] Updated weights for policy 1, policy_version 41520 (0.0010) -[2023-10-15 16:27:04,775][52866] Updated weights for policy 1, policy_version 41530 (0.0010) -[2023-10-15 16:27:06,846][52833] Updated weights for policy 0, policy_version 41410 (0.0010) -[2023-10-15 16:27:07,215][52833] Updated weights for policy 0, policy_version 41420 (0.0008) -[2023-10-15 16:27:07,581][52833] Updated weights for policy 0, policy_version 41430 (0.0009) -[2023-10-15 16:27:07,948][52833] Updated weights for policy 0, policy_version 41440 (0.0008) -[2023-10-15 16:27:08,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 84967424. Throughput: 0: 1805.8, 1: 1792.0. Samples: 21249380. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) -[2023-10-15 16:27:08,442][51532] Avg episode reward: [(0, '43.960'), (1, '43.690')] -[2023-10-15 16:27:08,517][52866] Updated weights for policy 1, policy_version 41540 (0.0008) -[2023-10-15 16:27:08,883][52866] Updated weights for policy 1, policy_version 41550 (0.0007) -[2023-10-15 16:27:09,249][52866] Updated weights for policy 1, policy_version 41560 (0.0008) -[2023-10-15 16:27:11,725][52833] Updated weights for policy 0, policy_version 41450 (0.0008) -[2023-10-15 16:27:12,099][52833] Updated weights for policy 0, policy_version 41460 (0.0007) -[2023-10-15 16:27:12,466][52833] Updated weights for policy 0, policy_version 41470 (0.0009) -[2023-10-15 16:27:13,058][52866] Updated weights for policy 1, policy_version 41570 (0.0008) -[2023-10-15 16:27:13,426][52866] Updated weights for policy 1, policy_version 41580 (0.0008) -[2023-10-15 16:27:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85032960. Throughput: 0: 1778.5, 1: 1809.0. Samples: 21270278. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) -[2023-10-15 16:27:13,442][51532] Avg episode reward: [(0, '44.710'), (1, '45.300')] -[2023-10-15 16:27:13,804][52866] Updated weights for policy 1, policy_version 41590 (0.0007) -[2023-10-15 16:27:14,163][52866] Updated weights for policy 1, policy_version 41600 (0.0008) -[2023-10-15 16:27:16,260][52833] Updated weights for policy 0, policy_version 41480 (0.0008) -[2023-10-15 16:27:16,645][52833] Updated weights for policy 0, policy_version 41490 (0.0009) -[2023-10-15 16:27:17,014][52833] Updated weights for policy 0, policy_version 41500 (0.0010) -[2023-10-15 16:27:17,828][52866] Updated weights for policy 1, policy_version 41610 (0.0007) -[2023-10-15 16:27:18,192][52866] Updated weights for policy 1, policy_version 41620 (0.0008) -[2023-10-15 16:27:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85098496. Throughput: 0: 1802.7, 1: 1785.8. Samples: 21281604. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) -[2023-10-15 16:27:18,442][51532] Avg episode reward: [(0, '47.290'), (1, '44.680')] -[2023-10-15 16:27:18,558][52866] Updated weights for policy 1, policy_version 41630 (0.0008) -[2023-10-15 16:27:20,716][52833] Updated weights for policy 0, policy_version 41510 (0.0008) -[2023-10-15 16:27:21,087][52833] Updated weights for policy 0, policy_version 41520 (0.0008) -[2023-10-15 16:27:21,458][52833] Updated weights for policy 0, policy_version 41530 (0.0008) -[2023-10-15 16:27:22,161][52866] Updated weights for policy 1, policy_version 41640 (0.0008) -[2023-10-15 16:27:22,533][52866] Updated weights for policy 1, policy_version 41650 (0.0007) -[2023-10-15 16:27:22,901][52866] Updated weights for policy 1, policy_version 41660 (0.0007) -[2023-10-15 16:27:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 85196800. Throughput: 0: 1779.4, 1: 1801.4. Samples: 21302472. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) -[2023-10-15 16:27:23,441][51532] Avg episode reward: [(0, '46.290'), (1, '44.200')] -[2023-10-15 16:27:25,180][52833] Updated weights for policy 0, policy_version 41540 (0.0008) -[2023-10-15 16:27:25,561][52833] Updated weights for policy 0, policy_version 41550 (0.0009) -[2023-10-15 16:27:25,924][52833] Updated weights for policy 0, policy_version 41560 (0.0008) -[2023-10-15 16:27:26,697][52866] Updated weights for policy 1, policy_version 41670 (0.0008) -[2023-10-15 16:27:27,069][52866] Updated weights for policy 1, policy_version 41680 (0.0008) -[2023-10-15 16:27:27,436][52866] Updated weights for policy 1, policy_version 41690 (0.0007) -[2023-10-15 16:27:28,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 85262336. Throughput: 0: 1782.1, 1: 1789.9. Samples: 21323800. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) -[2023-10-15 16:27:28,442][51532] Avg episode reward: [(0, '44.510'), (1, '43.670')] -[2023-10-15 16:27:29,710][52833] Updated weights for policy 0, policy_version 41570 (0.0008) -[2023-10-15 16:27:30,079][52833] Updated weights for policy 0, policy_version 41580 (0.0008) -[2023-10-15 16:27:30,454][52833] Updated weights for policy 0, policy_version 41590 (0.0007) -[2023-10-15 16:27:30,815][52833] Updated weights for policy 0, policy_version 41600 (0.0007) -[2023-10-15 16:27:31,041][52866] Updated weights for policy 1, policy_version 41700 (0.0008) -[2023-10-15 16:27:31,411][52866] Updated weights for policy 1, policy_version 41710 (0.0009) -[2023-10-15 16:27:31,779][52866] Updated weights for policy 1, policy_version 41720 (0.0010) -[2023-10-15 16:27:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 85327872. Throughput: 0: 1777.9, 1: 1808.3. Samples: 21335076. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) -[2023-10-15 16:27:33,442][51532] Avg episode reward: [(0, '45.500'), (1, '42.680')] -[2023-10-15 16:27:34,425][52833] Updated weights for policy 0, policy_version 41610 (0.0007) -[2023-10-15 16:27:34,794][52833] Updated weights for policy 0, policy_version 41620 (0.0008) -[2023-10-15 16:27:35,170][52833] Updated weights for policy 0, policy_version 41630 (0.0007) -[2023-10-15 16:27:35,594][52866] Updated weights for policy 1, policy_version 41730 (0.0008) -[2023-10-15 16:27:35,954][52866] Updated weights for policy 1, policy_version 41740 (0.0009) -[2023-10-15 16:27:36,321][52866] Updated weights for policy 1, policy_version 41750 (0.0010) -[2023-10-15 16:27:36,685][52866] Updated weights for policy 1, policy_version 41760 (0.0009) -[2023-10-15 16:27:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 85393408. Throughput: 0: 1786.0, 1: 1795.0. Samples: 21356218. Policy #0 lag: (min: 31.0, avg: 32.2, max: 57.0) -[2023-10-15 16:27:38,442][51532] Avg episode reward: [(0, '44.230'), (1, '44.180')] -[2023-10-15 16:27:38,968][52833] Updated weights for policy 0, policy_version 41640 (0.0008) -[2023-10-15 16:27:39,341][52833] Updated weights for policy 0, policy_version 41650 (0.0010) -[2023-10-15 16:27:39,711][52833] Updated weights for policy 0, policy_version 41660 (0.0011) -[2023-10-15 16:27:40,614][52866] Updated weights for policy 1, policy_version 41770 (0.0008) -[2023-10-15 16:27:40,976][52866] Updated weights for policy 1, policy_version 41780 (0.0007) -[2023-10-15 16:27:41,332][52866] Updated weights for policy 1, policy_version 41790 (0.0009) -[2023-10-15 16:27:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 85458944. Throughput: 0: 1791.3, 1: 1795.6. Samples: 21378560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:27:43,442][51532] Avg episode reward: [(0, '41.720'), (1, '47.500')] -[2023-10-15 16:27:43,449][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000041792_42795008.pth... -[2023-10-15 16:27:43,480][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000040096_41058304.pth -[2023-10-15 16:27:43,484][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000041792_42795008.pth -[2023-10-15 16:27:43,546][52833] Updated weights for policy 0, policy_version 41670 (0.0008) -[2023-10-15 16:27:43,924][52833] Updated weights for policy 0, policy_version 41680 (0.0010) -[2023-10-15 16:27:44,295][52833] Updated weights for policy 0, policy_version 41690 (0.0008) -[2023-10-15 16:27:44,519][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000041696_42696704.pth... -[2023-10-15 16:27:44,549][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000040000_40960000.pth -[2023-10-15 16:27:44,553][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000041696_42696704.pth -[2023-10-15 16:27:45,072][52866] Updated weights for policy 1, policy_version 41800 (0.0008) -[2023-10-15 16:27:45,449][52866] Updated weights for policy 1, policy_version 41810 (0.0010) -[2023-10-15 16:27:45,812][52866] Updated weights for policy 1, policy_version 41820 (0.0007) -[2023-10-15 16:27:48,150][52833] Updated weights for policy 0, policy_version 41700 (0.0009) -[2023-10-15 16:27:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 85524480. Throughput: 0: 1787.1, 1: 1796.4. Samples: 21388450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:27:48,442][51532] Avg episode reward: [(0, '42.010'), (1, '47.400')] -[2023-10-15 16:27:48,527][52833] Updated weights for policy 0, policy_version 41710 (0.0008) -[2023-10-15 16:27:48,892][52833] Updated weights for policy 0, policy_version 41720 (0.0008) -[2023-10-15 16:27:49,622][52866] Updated weights for policy 1, policy_version 41830 (0.0009) -[2023-10-15 16:27:49,990][52866] Updated weights for policy 1, policy_version 41840 (0.0011) -[2023-10-15 16:27:50,364][52866] Updated weights for policy 1, policy_version 41850 (0.0009) -[2023-10-15 16:27:52,652][52833] Updated weights for policy 0, policy_version 41730 (0.0007) -[2023-10-15 16:27:53,019][52833] Updated weights for policy 0, policy_version 41740 (0.0008) -[2023-10-15 16:27:53,392][52833] Updated weights for policy 0, policy_version 41750 (0.0009) -[2023-10-15 16:27:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 85590016. Throughput: 0: 1783.2, 1: 1794.3. Samples: 21410370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:27:53,442][51532] Avg episode reward: [(0, '41.440'), (1, '45.810')] -[2023-10-15 16:27:53,759][52833] Updated weights for policy 0, policy_version 41760 (0.0008) -[2023-10-15 16:27:54,109][52866] Updated weights for policy 1, policy_version 41860 (0.0010) -[2023-10-15 16:27:54,502][52866] Updated weights for policy 1, policy_version 41870 (0.0010) -[2023-10-15 16:27:54,858][52866] Updated weights for policy 1, policy_version 41880 (0.0011) -[2023-10-15 16:27:57,576][52833] Updated weights for policy 0, policy_version 41770 (0.0009) -[2023-10-15 16:27:57,937][52833] Updated weights for policy 0, policy_version 41780 (0.0008) -[2023-10-15 16:27:58,292][52833] Updated weights for policy 0, policy_version 41790 (0.0008) -[2023-10-15 16:27:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 85688320. Throughput: 0: 1802.1, 1: 1799.7. Samples: 21432362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:27:58,441][51532] Avg episode reward: [(0, '44.100'), (1, '43.400')] -[2023-10-15 16:27:58,537][52866] Updated weights for policy 1, policy_version 41890 (0.0008) -[2023-10-15 16:27:58,906][52866] Updated weights for policy 1, policy_version 41900 (0.0008) -[2023-10-15 16:27:59,278][52866] Updated weights for policy 1, policy_version 41910 (0.0009) -[2023-10-15 16:27:59,644][52866] Updated weights for policy 1, policy_version 41920 (0.0010) -[2023-10-15 16:28:02,123][52833] Updated weights for policy 0, policy_version 41800 (0.0008) -[2023-10-15 16:28:02,484][52833] Updated weights for policy 0, policy_version 41810 (0.0007) -[2023-10-15 16:28:02,852][52833] Updated weights for policy 0, policy_version 41820 (0.0008) -[2023-10-15 16:28:03,323][52866] Updated weights for policy 1, policy_version 41930 (0.0007) -[2023-10-15 16:28:03,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85753856. Throughput: 0: 1786.1, 1: 1801.6. Samples: 21443054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:03,442][51532] Avg episode reward: [(0, '43.310'), (1, '44.360')] -[2023-10-15 16:28:03,696][52866] Updated weights for policy 1, policy_version 41940 (0.0008) -[2023-10-15 16:28:04,057][52866] Updated weights for policy 1, policy_version 41950 (0.0009) -[2023-10-15 16:28:06,417][52833] Updated weights for policy 0, policy_version 41830 (0.0009) -[2023-10-15 16:28:06,784][52833] Updated weights for policy 0, policy_version 41840 (0.0008) -[2023-10-15 16:28:07,143][52833] Updated weights for policy 0, policy_version 41850 (0.0009) -[2023-10-15 16:28:07,793][52866] Updated weights for policy 1, policy_version 41960 (0.0008) -[2023-10-15 16:28:08,167][52866] Updated weights for policy 1, policy_version 41970 (0.0009) -[2023-10-15 16:28:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 85819392. Throughput: 0: 1802.3, 1: 1800.8. Samples: 21464610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:08,441][51532] Avg episode reward: [(0, '45.200'), (1, '44.680')] -[2023-10-15 16:28:08,527][52866] Updated weights for policy 1, policy_version 41980 (0.0008) -[2023-10-15 16:28:10,774][52833] Updated weights for policy 0, policy_version 41860 (0.0008) -[2023-10-15 16:28:11,135][52833] Updated weights for policy 0, policy_version 41870 (0.0007) -[2023-10-15 16:28:11,502][52833] Updated weights for policy 0, policy_version 41880 (0.0008) -[2023-10-15 16:28:12,086][52866] Updated weights for policy 1, policy_version 41990 (0.0007) -[2023-10-15 16:28:12,457][52866] Updated weights for policy 1, policy_version 42000 (0.0007) -[2023-10-15 16:28:12,825][52866] Updated weights for policy 1, policy_version 42010 (0.0007) -[2023-10-15 16:28:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 85917696. Throughput: 0: 1790.2, 1: 1801.5. Samples: 21485426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:13,442][51532] Avg episode reward: [(0, '45.130'), (1, '44.030')] -[2023-10-15 16:28:15,238][52833] Updated weights for policy 0, policy_version 41890 (0.0008) -[2023-10-15 16:28:15,614][52833] Updated weights for policy 0, policy_version 41900 (0.0007) -[2023-10-15 16:28:15,986][52833] Updated weights for policy 0, policy_version 41910 (0.0009) -[2023-10-15 16:28:16,354][52833] Updated weights for policy 0, policy_version 41920 (0.0009) -[2023-10-15 16:28:16,606][52866] Updated weights for policy 1, policy_version 42020 (0.0007) -[2023-10-15 16:28:16,965][52866] Updated weights for policy 1, policy_version 42030 (0.0008) -[2023-10-15 16:28:17,332][52866] Updated weights for policy 1, policy_version 42040 (0.0009) -[2023-10-15 16:28:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 85983232. Throughput: 0: 1807.3, 1: 1796.0. Samples: 21497224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:18,442][51532] Avg episode reward: [(0, '43.150'), (1, '43.270')] -[2023-10-15 16:28:20,105][52833] Updated weights for policy 0, policy_version 41930 (0.0007) -[2023-10-15 16:28:20,472][52833] Updated weights for policy 0, policy_version 41940 (0.0007) -[2023-10-15 16:28:20,836][52833] Updated weights for policy 0, policy_version 41950 (0.0010) -[2023-10-15 16:28:21,047][52866] Updated weights for policy 1, policy_version 42050 (0.0010) -[2023-10-15 16:28:21,421][52866] Updated weights for policy 1, policy_version 42060 (0.0010) -[2023-10-15 16:28:21,791][52866] Updated weights for policy 1, policy_version 42070 (0.0008) -[2023-10-15 16:28:22,155][52866] Updated weights for policy 1, policy_version 42080 (0.0007) -[2023-10-15 16:28:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 86048768. Throughput: 0: 1790.1, 1: 1806.3. Samples: 21518058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:23,441][51532] Avg episode reward: [(0, '45.350'), (1, '44.160')] -[2023-10-15 16:28:24,487][52833] Updated weights for policy 0, policy_version 41960 (0.0008) -[2023-10-15 16:28:24,857][52833] Updated weights for policy 0, policy_version 41970 (0.0007) -[2023-10-15 16:28:25,225][52833] Updated weights for policy 0, policy_version 41980 (0.0009) -[2023-10-15 16:28:25,949][52866] Updated weights for policy 1, policy_version 42090 (0.0007) -[2023-10-15 16:28:26,322][52866] Updated weights for policy 1, policy_version 42100 (0.0007) -[2023-10-15 16:28:26,690][52866] Updated weights for policy 1, policy_version 42110 (0.0007) -[2023-10-15 16:28:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 86114304. Throughput: 0: 1790.8, 1: 1800.3. Samples: 21540158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:28,442][51532] Avg episode reward: [(0, '44.240'), (1, '48.810')] -[2023-10-15 16:28:29,252][52833] Updated weights for policy 0, policy_version 41990 (0.0009) -[2023-10-15 16:28:29,611][52833] Updated weights for policy 0, policy_version 42000 (0.0007) -[2023-10-15 16:28:29,985][52833] Updated weights for policy 0, policy_version 42010 (0.0008) -[2023-10-15 16:28:30,415][52866] Updated weights for policy 1, policy_version 42120 (0.0007) -[2023-10-15 16:28:30,788][52866] Updated weights for policy 1, policy_version 42130 (0.0009) -[2023-10-15 16:28:31,162][52866] Updated weights for policy 1, policy_version 42140 (0.0009) -[2023-10-15 16:28:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 86179840. Throughput: 0: 1791.4, 1: 1809.2. Samples: 21550478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:33,442][51532] Avg episode reward: [(0, '43.580'), (1, '45.580')] -[2023-10-15 16:28:33,834][52833] Updated weights for policy 0, policy_version 42020 (0.0009) -[2023-10-15 16:28:34,213][52833] Updated weights for policy 0, policy_version 42030 (0.0012) -[2023-10-15 16:28:34,571][52833] Updated weights for policy 0, policy_version 42040 (0.0007) -[2023-10-15 16:28:34,870][52866] Updated weights for policy 1, policy_version 42150 (0.0008) -[2023-10-15 16:28:35,241][52866] Updated weights for policy 1, policy_version 42160 (0.0008) -[2023-10-15 16:28:35,596][52866] Updated weights for policy 1, policy_version 42170 (0.0007) -[2023-10-15 16:28:38,287][52833] Updated weights for policy 0, policy_version 42050 (0.0008) -[2023-10-15 16:28:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 86245376. Throughput: 0: 1799.2, 1: 1799.7. Samples: 21572322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:38,442][51532] Avg episode reward: [(0, '44.850'), (1, '43.690')] -[2023-10-15 16:28:38,661][52833] Updated weights for policy 0, policy_version 42060 (0.0008) -[2023-10-15 16:28:39,029][52833] Updated weights for policy 0, policy_version 42070 (0.0010) -[2023-10-15 16:28:39,397][52833] Updated weights for policy 0, policy_version 42080 (0.0008) -[2023-10-15 16:28:39,413][52866] Updated weights for policy 1, policy_version 42180 (0.0008) -[2023-10-15 16:28:39,783][52866] Updated weights for policy 1, policy_version 42190 (0.0009) -[2023-10-15 16:28:40,145][52866] Updated weights for policy 1, policy_version 42200 (0.0008) -[2023-10-15 16:28:43,079][52833] Updated weights for policy 0, policy_version 42090 (0.0010) -[2023-10-15 16:28:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 86310912. Throughput: 0: 1810.2, 1: 1795.2. Samples: 21594602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:43,441][51532] Avg episode reward: [(0, '44.770'), (1, '46.190')] -[2023-10-15 16:28:43,464][52833] Updated weights for policy 0, policy_version 42100 (0.0009) -[2023-10-15 16:28:43,834][52833] Updated weights for policy 0, policy_version 42110 (0.0008) -[2023-10-15 16:28:43,844][52866] Updated weights for policy 1, policy_version 42210 (0.0010) -[2023-10-15 16:28:44,210][52866] Updated weights for policy 1, policy_version 42220 (0.0007) -[2023-10-15 16:28:44,569][52866] Updated weights for policy 1, policy_version 42230 (0.0008) -[2023-10-15 16:28:44,938][52866] Updated weights for policy 1, policy_version 42240 (0.0008) -[2023-10-15 16:28:47,476][52833] Updated weights for policy 0, policy_version 42120 (0.0008) -[2023-10-15 16:28:47,851][52833] Updated weights for policy 0, policy_version 42130 (0.0008) -[2023-10-15 16:28:48,215][52833] Updated weights for policy 0, policy_version 42140 (0.0008) -[2023-10-15 16:28:48,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 86409216. Throughput: 0: 1794.3, 1: 1798.2. Samples: 21604716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:48,442][51532] Avg episode reward: [(0, '42.390'), (1, '44.800')] -[2023-10-15 16:28:48,715][52866] Updated weights for policy 1, policy_version 42250 (0.0007) -[2023-10-15 16:28:49,074][52866] Updated weights for policy 1, policy_version 42260 (0.0007) -[2023-10-15 16:28:49,453][52866] Updated weights for policy 1, policy_version 42270 (0.0008) -[2023-10-15 16:28:52,057][52833] Updated weights for policy 0, policy_version 42150 (0.0008) -[2023-10-15 16:28:52,421][52833] Updated weights for policy 0, policy_version 42160 (0.0009) -[2023-10-15 16:28:52,792][52833] Updated weights for policy 0, policy_version 42170 (0.0008) -[2023-10-15 16:28:53,096][52866] Updated weights for policy 1, policy_version 42280 (0.0008) -[2023-10-15 16:28:53,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 86474752. Throughput: 0: 1812.3, 1: 1800.3. Samples: 21627180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:53,442][51532] Avg episode reward: [(0, '42.060'), (1, '43.530')] -[2023-10-15 16:28:53,473][52866] Updated weights for policy 1, policy_version 42290 (0.0008) -[2023-10-15 16:28:53,834][52866] Updated weights for policy 1, policy_version 42300 (0.0009) -[2023-10-15 16:28:56,617][52833] Updated weights for policy 0, policy_version 42180 (0.0009) -[2023-10-15 16:28:56,996][52833] Updated weights for policy 0, policy_version 42190 (0.0010) -[2023-10-15 16:28:57,355][52833] Updated weights for policy 0, policy_version 42200 (0.0010) -[2023-10-15 16:28:57,611][52866] Updated weights for policy 1, policy_version 42310 (0.0009) -[2023-10-15 16:28:57,978][52866] Updated weights for policy 1, policy_version 42320 (0.0010) -[2023-10-15 16:28:58,355][52866] Updated weights for policy 1, policy_version 42330 (0.0009) -[2023-10-15 16:28:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 86540288. Throughput: 0: 1785.3, 1: 1814.4. Samples: 21647416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:28:58,442][51532] Avg episode reward: [(0, '44.090'), (1, '44.930')] -[2023-10-15 16:29:00,879][52833] Updated weights for policy 0, policy_version 42210 (0.0008) -[2023-10-15 16:29:01,248][52833] Updated weights for policy 0, policy_version 42220 (0.0008) -[2023-10-15 16:29:01,627][52833] Updated weights for policy 0, policy_version 42230 (0.0008) -[2023-10-15 16:29:01,992][52833] Updated weights for policy 0, policy_version 42240 (0.0008) -[2023-10-15 16:29:02,067][52866] Updated weights for policy 1, policy_version 42340 (0.0011) -[2023-10-15 16:29:02,434][52866] Updated weights for policy 1, policy_version 42350 (0.0009) -[2023-10-15 16:29:02,809][52866] Updated weights for policy 1, policy_version 42360 (0.0009) -[2023-10-15 16:29:03,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 86638592. Throughput: 0: 1804.3, 1: 1799.9. Samples: 21659412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:29:03,442][51532] Avg episode reward: [(0, '43.460'), (1, '45.760')] -[2023-10-15 16:29:05,758][52833] Updated weights for policy 0, policy_version 42250 (0.0007) -[2023-10-15 16:29:06,125][52833] Updated weights for policy 0, policy_version 42260 (0.0009) -[2023-10-15 16:29:06,492][52833] Updated weights for policy 0, policy_version 42270 (0.0007) -[2023-10-15 16:29:06,537][52866] Updated weights for policy 1, policy_version 42370 (0.0008) -[2023-10-15 16:29:06,892][52866] Updated weights for policy 1, policy_version 42380 (0.0009) -[2023-10-15 16:29:07,260][52866] Updated weights for policy 1, policy_version 42390 (0.0007) -[2023-10-15 16:29:07,629][52866] Updated weights for policy 1, policy_version 42400 (0.0008) -[2023-10-15 16:29:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 86704128. Throughput: 0: 1787.1, 1: 1809.3. Samples: 21679896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:29:08,442][51532] Avg episode reward: [(0, '42.300'), (1, '46.720')] -[2023-10-15 16:29:10,238][52833] Updated weights for policy 0, policy_version 42280 (0.0010) -[2023-10-15 16:29:10,602][52833] Updated weights for policy 0, policy_version 42290 (0.0009) -[2023-10-15 16:29:10,970][52833] Updated weights for policy 0, policy_version 42300 (0.0008) -[2023-10-15 16:29:11,336][52866] Updated weights for policy 1, policy_version 42410 (0.0009) -[2023-10-15 16:29:11,697][52866] Updated weights for policy 1, policy_version 42420 (0.0010) -[2023-10-15 16:29:12,063][52866] Updated weights for policy 1, policy_version 42430 (0.0009) -[2023-10-15 16:29:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 86769664. Throughput: 0: 1790.9, 1: 1796.6. Samples: 21701596. Policy #0 lag: (min: 9.0, avg: 13.0, max: 41.0) -[2023-10-15 16:29:13,441][51532] Avg episode reward: [(0, '42.760'), (1, '43.900')] -[2023-10-15 16:29:14,792][52833] Updated weights for policy 0, policy_version 42310 (0.0007) -[2023-10-15 16:29:15,162][52833] Updated weights for policy 0, policy_version 42320 (0.0008) -[2023-10-15 16:29:15,532][52833] Updated weights for policy 0, policy_version 42330 (0.0009) -[2023-10-15 16:29:15,865][52866] Updated weights for policy 1, policy_version 42440 (0.0008) -[2023-10-15 16:29:16,232][52866] Updated weights for policy 1, policy_version 42450 (0.0007) -[2023-10-15 16:29:16,603][52866] Updated weights for policy 1, policy_version 42460 (0.0009) -[2023-10-15 16:29:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 86835200. Throughput: 0: 1788.6, 1: 1806.9. Samples: 21712278. Policy #0 lag: (min: 9.0, avg: 13.0, max: 41.0) -[2023-10-15 16:29:18,442][51532] Avg episode reward: [(0, '44.090'), (1, '42.790')] -[2023-10-15 16:29:19,246][52833] Updated weights for policy 0, policy_version 42340 (0.0007) -[2023-10-15 16:29:19,616][52833] Updated weights for policy 0, policy_version 42350 (0.0011) -[2023-10-15 16:29:19,986][52833] Updated weights for policy 0, policy_version 42360 (0.0009) -[2023-10-15 16:29:20,468][52866] Updated weights for policy 1, policy_version 42470 (0.0009) -[2023-10-15 16:29:20,824][52866] Updated weights for policy 1, policy_version 42480 (0.0008) -[2023-10-15 16:29:21,188][52866] Updated weights for policy 1, policy_version 42490 (0.0009) -[2023-10-15 16:29:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 86900736. Throughput: 0: 1790.9, 1: 1799.4. Samples: 21733884. Policy #0 lag: (min: 9.0, avg: 13.0, max: 41.0) -[2023-10-15 16:29:23,441][51532] Avg episode reward: [(0, '40.620'), (1, '44.240')] -[2023-10-15 16:29:23,626][52833] Updated weights for policy 0, policy_version 42370 (0.0008) -[2023-10-15 16:29:24,007][52833] Updated weights for policy 0, policy_version 42380 (0.0010) -[2023-10-15 16:29:24,372][52833] Updated weights for policy 0, policy_version 42390 (0.0007) -[2023-10-15 16:29:24,746][52833] Updated weights for policy 0, policy_version 42400 (0.0010) -[2023-10-15 16:29:24,977][52866] Updated weights for policy 1, policy_version 42500 (0.0007) -[2023-10-15 16:29:25,338][52866] Updated weights for policy 1, policy_version 42510 (0.0008) -[2023-10-15 16:29:25,702][52866] Updated weights for policy 1, policy_version 42520 (0.0008) -[2023-10-15 16:29:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 86966272. Throughput: 0: 1797.5, 1: 1800.5. Samples: 21756512. Policy #0 lag: (min: 9.0, avg: 13.0, max: 41.0) -[2023-10-15 16:29:28,442][51532] Avg episode reward: [(0, '41.240'), (1, '41.350')] -[2023-10-15 16:29:28,562][52833] Updated weights for policy 0, policy_version 42410 (0.0008) -[2023-10-15 16:29:28,930][52833] Updated weights for policy 0, policy_version 42420 (0.0008) -[2023-10-15 16:29:29,304][52833] Updated weights for policy 0, policy_version 42430 (0.0009) -[2023-10-15 16:29:29,310][52866] Updated weights for policy 1, policy_version 42530 (0.0011) -[2023-10-15 16:29:29,681][52866] Updated weights for policy 1, policy_version 42540 (0.0009) -[2023-10-15 16:29:30,058][52866] Updated weights for policy 1, policy_version 42550 (0.0008) -[2023-10-15 16:29:30,419][52866] Updated weights for policy 1, policy_version 42560 (0.0007) -[2023-10-15 16:29:33,055][52833] Updated weights for policy 0, policy_version 42440 (0.0010) -[2023-10-15 16:29:33,438][52833] Updated weights for policy 0, policy_version 42450 (0.0010) -[2023-10-15 16:29:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 87031808. Throughput: 0: 1792.2, 1: 1802.1. Samples: 21766460. Policy #0 lag: (min: 9.0, avg: 13.0, max: 41.0) -[2023-10-15 16:29:33,442][51532] Avg episode reward: [(0, '43.170'), (1, '42.720')] -[2023-10-15 16:29:33,804][52833] Updated weights for policy 0, policy_version 42460 (0.0009) -[2023-10-15 16:29:34,080][52866] Updated weights for policy 1, policy_version 42570 (0.0008) -[2023-10-15 16:29:34,453][52866] Updated weights for policy 1, policy_version 42580 (0.0008) -[2023-10-15 16:29:34,812][52866] Updated weights for policy 1, policy_version 42590 (0.0011) -[2023-10-15 16:29:37,472][52833] Updated weights for policy 0, policy_version 42470 (0.0008) -[2023-10-15 16:29:37,840][52833] Updated weights for policy 0, policy_version 42480 (0.0008) -[2023-10-15 16:29:38,211][52833] Updated weights for policy 0, policy_version 42490 (0.0007) -[2023-10-15 16:29:38,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 87130112. Throughput: 0: 1794.6, 1: 1796.7. Samples: 21788790. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:29:38,441][51532] Avg episode reward: [(0, '45.470'), (1, '42.820')] -[2023-10-15 16:29:38,562][52866] Updated weights for policy 1, policy_version 42600 (0.0008) -[2023-10-15 16:29:38,926][52866] Updated weights for policy 1, policy_version 42610 (0.0007) -[2023-10-15 16:29:39,294][52866] Updated weights for policy 1, policy_version 42620 (0.0008) -[2023-10-15 16:29:41,996][52833] Updated weights for policy 0, policy_version 42500 (0.0008) -[2023-10-15 16:29:42,376][52833] Updated weights for policy 0, policy_version 42510 (0.0009) -[2023-10-15 16:29:42,747][52833] Updated weights for policy 0, policy_version 42520 (0.0007) -[2023-10-15 16:29:43,074][52866] Updated weights for policy 1, policy_version 42630 (0.0008) -[2023-10-15 16:29:43,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 87195648. Throughput: 0: 1803.4, 1: 1805.3. Samples: 21809806. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:29:43,441][51532] Avg episode reward: [(0, '44.250'), (1, '41.970')] -[2023-10-15 16:29:43,447][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000042528_43548672.pth... -[2023-10-15 16:29:43,451][52866] Updated weights for policy 1, policy_version 42640 (0.0009) -[2023-10-15 16:29:43,487][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000040832_41811968.pth -[2023-10-15 16:29:43,821][52866] Updated weights for policy 1, policy_version 42650 (0.0008) -[2023-10-15 16:29:44,029][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000042656_43679744.pth... -[2023-10-15 16:29:44,070][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000040960_41943040.pth -[2023-10-15 16:29:46,499][52833] Updated weights for policy 0, policy_version 42530 (0.0008) -[2023-10-15 16:29:46,878][52833] Updated weights for policy 0, policy_version 42540 (0.0010) -[2023-10-15 16:29:47,252][52833] Updated weights for policy 0, policy_version 42550 (0.0008) -[2023-10-15 16:29:47,529][52866] Updated weights for policy 1, policy_version 42660 (0.0007) -[2023-10-15 16:29:47,624][52833] Updated weights for policy 0, policy_version 42560 (0.0008) -[2023-10-15 16:29:47,905][52866] Updated weights for policy 1, policy_version 42670 (0.0011) -[2023-10-15 16:29:48,274][52866] Updated weights for policy 1, policy_version 42680 (0.0008) -[2023-10-15 16:29:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87261184. Throughput: 0: 1797.3, 1: 1794.4. Samples: 21821036. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:29:48,442][51532] Avg episode reward: [(0, '46.180'), (1, '41.550')] -[2023-10-15 16:29:51,262][52833] Updated weights for policy 0, policy_version 42570 (0.0008) -[2023-10-15 16:29:51,634][52833] Updated weights for policy 0, policy_version 42580 (0.0010) -[2023-10-15 16:29:51,997][52833] Updated weights for policy 0, policy_version 42590 (0.0009) -[2023-10-15 16:29:52,047][52866] Updated weights for policy 1, policy_version 42690 (0.0010) -[2023-10-15 16:29:52,414][52866] Updated weights for policy 1, policy_version 42700 (0.0007) -[2023-10-15 16:29:52,788][52866] Updated weights for policy 1, policy_version 42710 (0.0010) -[2023-10-15 16:29:53,151][52866] Updated weights for policy 1, policy_version 42720 (0.0007) -[2023-10-15 16:29:53,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 87359488. Throughput: 0: 1804.3, 1: 1804.6. Samples: 21842296. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:29:53,442][51532] Avg episode reward: [(0, '46.530'), (1, '41.500')] -[2023-10-15 16:29:55,792][52833] Updated weights for policy 0, policy_version 42600 (0.0008) -[2023-10-15 16:29:56,157][52833] Updated weights for policy 0, policy_version 42610 (0.0007) -[2023-10-15 16:29:56,521][52833] Updated weights for policy 0, policy_version 42620 (0.0008) -[2023-10-15 16:29:56,945][52866] Updated weights for policy 1, policy_version 42730 (0.0007) -[2023-10-15 16:29:57,303][52866] Updated weights for policy 1, policy_version 42740 (0.0009) -[2023-10-15 16:29:57,676][52866] Updated weights for policy 1, policy_version 42750 (0.0008) -[2023-10-15 16:29:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 87425024. Throughput: 0: 1797.6, 1: 1787.8. Samples: 21862938. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:29:58,441][51532] Avg episode reward: [(0, '43.710'), (1, '39.690')] -[2023-10-15 16:30:00,279][52833] Updated weights for policy 0, policy_version 42630 (0.0010) -[2023-10-15 16:30:00,652][52833] Updated weights for policy 0, policy_version 42640 (0.0008) -[2023-10-15 16:30:01,017][52833] Updated weights for policy 0, policy_version 42650 (0.0008) -[2023-10-15 16:30:01,389][52866] Updated weights for policy 1, policy_version 42760 (0.0009) -[2023-10-15 16:30:01,768][52866] Updated weights for policy 1, policy_version 42770 (0.0011) -[2023-10-15 16:30:02,134][52866] Updated weights for policy 1, policy_version 42780 (0.0007) -[2023-10-15 16:30:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 87490560. Throughput: 0: 1808.4, 1: 1805.9. Samples: 21874922. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:30:03,442][51532] Avg episode reward: [(0, '41.780'), (1, '42.140')] -[2023-10-15 16:30:04,730][52833] Updated weights for policy 0, policy_version 42660 (0.0008) -[2023-10-15 16:30:05,098][52833] Updated weights for policy 0, policy_version 42670 (0.0007) -[2023-10-15 16:30:05,473][52833] Updated weights for policy 0, policy_version 42680 (0.0007) -[2023-10-15 16:30:05,956][52866] Updated weights for policy 1, policy_version 42790 (0.0010) -[2023-10-15 16:30:06,319][52866] Updated weights for policy 1, policy_version 42800 (0.0007) -[2023-10-15 16:30:06,692][52866] Updated weights for policy 1, policy_version 42810 (0.0009) -[2023-10-15 16:30:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 87556096. Throughput: 0: 1800.3, 1: 1797.5. Samples: 21895786. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:30:08,442][51532] Avg episode reward: [(0, '46.040'), (1, '43.110')] -[2023-10-15 16:30:09,198][52833] Updated weights for policy 0, policy_version 42690 (0.0008) -[2023-10-15 16:30:09,577][52833] Updated weights for policy 0, policy_version 42700 (0.0007) -[2023-10-15 16:30:09,943][52833] Updated weights for policy 0, policy_version 42710 (0.0008) -[2023-10-15 16:30:10,321][52833] Updated weights for policy 0, policy_version 42720 (0.0008) -[2023-10-15 16:30:10,405][52866] Updated weights for policy 1, policy_version 42820 (0.0007) -[2023-10-15 16:30:10,798][52866] Updated weights for policy 1, policy_version 42830 (0.0007) -[2023-10-15 16:30:11,159][52866] Updated weights for policy 1, policy_version 42840 (0.0010) -[2023-10-15 16:30:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 87621632. Throughput: 0: 1799.0, 1: 1792.4. Samples: 21918126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:30:13,442][51532] Avg episode reward: [(0, '48.820'), (1, '44.860')] -[2023-10-15 16:30:13,452][52410] Saving new best policy, reward=48.820! -[2023-10-15 16:30:14,038][52833] Updated weights for policy 0, policy_version 42730 (0.0011) -[2023-10-15 16:30:14,407][52833] Updated weights for policy 0, policy_version 42740 (0.0009) -[2023-10-15 16:30:14,781][52833] Updated weights for policy 0, policy_version 42750 (0.0011) -[2023-10-15 16:30:14,955][52866] Updated weights for policy 1, policy_version 42850 (0.0009) -[2023-10-15 16:30:15,320][52866] Updated weights for policy 1, policy_version 42860 (0.0009) -[2023-10-15 16:30:15,694][52866] Updated weights for policy 1, policy_version 42870 (0.0011) -[2023-10-15 16:30:16,055][52866] Updated weights for policy 1, policy_version 42880 (0.0011) -[2023-10-15 16:30:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 87687168. Throughput: 0: 1798.6, 1: 1794.0. Samples: 21928130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:30:18,441][51532] Avg episode reward: [(0, '47.550'), (1, '43.720')] -[2023-10-15 16:30:18,561][52833] Updated weights for policy 0, policy_version 42760 (0.0009) -[2023-10-15 16:30:18,932][52833] Updated weights for policy 0, policy_version 42770 (0.0007) -[2023-10-15 16:30:19,290][52833] Updated weights for policy 0, policy_version 42780 (0.0008) -[2023-10-15 16:30:19,742][52866] Updated weights for policy 1, policy_version 42890 (0.0009) -[2023-10-15 16:30:20,113][52866] Updated weights for policy 1, policy_version 42900 (0.0009) -[2023-10-15 16:30:20,486][52866] Updated weights for policy 1, policy_version 42910 (0.0007) -[2023-10-15 16:30:22,802][52833] Updated weights for policy 0, policy_version 42790 (0.0009) -[2023-10-15 16:30:23,172][52833] Updated weights for policy 0, policy_version 42800 (0.0008) -[2023-10-15 16:30:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 87752704. Throughput: 0: 1796.7, 1: 1790.4. Samples: 21950210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:30:23,442][51532] Avg episode reward: [(0, '45.900'), (1, '45.980')] -[2023-10-15 16:30:23,529][52833] Updated weights for policy 0, policy_version 42810 (0.0009) -[2023-10-15 16:30:24,148][52866] Updated weights for policy 1, policy_version 42920 (0.0008) -[2023-10-15 16:30:24,516][52866] Updated weights for policy 1, policy_version 42930 (0.0008) -[2023-10-15 16:30:24,866][52866] Updated weights for policy 1, policy_version 42940 (0.0007) -[2023-10-15 16:30:27,253][52833] Updated weights for policy 0, policy_version 42820 (0.0009) -[2023-10-15 16:30:27,618][52833] Updated weights for policy 0, policy_version 42830 (0.0009) -[2023-10-15 16:30:27,987][52833] Updated weights for policy 0, policy_version 42840 (0.0007) -[2023-10-15 16:30:28,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 87851008. Throughput: 0: 1811.1, 1: 1798.4. Samples: 21972232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:30:28,442][51532] Avg episode reward: [(0, '47.040'), (1, '45.120')] -[2023-10-15 16:30:28,665][52866] Updated weights for policy 1, policy_version 42950 (0.0008) -[2023-10-15 16:30:29,031][52866] Updated weights for policy 1, policy_version 42960 (0.0007) -[2023-10-15 16:30:29,389][52866] Updated weights for policy 1, policy_version 42970 (0.0007) -[2023-10-15 16:30:31,856][52833] Updated weights for policy 0, policy_version 42850 (0.0007) -[2023-10-15 16:30:32,218][52833] Updated weights for policy 0, policy_version 42860 (0.0008) -[2023-10-15 16:30:32,588][52833] Updated weights for policy 0, policy_version 42870 (0.0008) -[2023-10-15 16:30:32,958][52833] Updated weights for policy 0, policy_version 42880 (0.0008) -[2023-10-15 16:30:33,133][52866] Updated weights for policy 1, policy_version 42980 (0.0008) -[2023-10-15 16:30:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 87916544. Throughput: 0: 1805.1, 1: 1793.2. Samples: 21982958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:30:33,442][51532] Avg episode reward: [(0, '44.930'), (1, '46.020')] -[2023-10-15 16:30:33,499][52866] Updated weights for policy 1, policy_version 42990 (0.0008) -[2023-10-15 16:30:33,871][52866] Updated weights for policy 1, policy_version 43000 (0.0010) -[2023-10-15 16:30:36,680][52833] Updated weights for policy 0, policy_version 42890 (0.0009) -[2023-10-15 16:30:37,044][52833] Updated weights for policy 0, policy_version 42900 (0.0008) -[2023-10-15 16:30:37,409][52833] Updated weights for policy 0, policy_version 42910 (0.0007) -[2023-10-15 16:30:37,718][52866] Updated weights for policy 1, policy_version 43010 (0.0009) -[2023-10-15 16:30:38,090][52866] Updated weights for policy 1, policy_version 43020 (0.0011) -[2023-10-15 16:30:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 87982080. Throughput: 0: 1812.8, 1: 1796.5. Samples: 22004714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:30:38,442][51532] Avg episode reward: [(0, '44.860'), (1, '45.160')] -[2023-10-15 16:30:38,462][52866] Updated weights for policy 1, policy_version 43030 (0.0008) -[2023-10-15 16:30:38,833][52866] Updated weights for policy 1, policy_version 43040 (0.0007) -[2023-10-15 16:30:41,116][52833] Updated weights for policy 0, policy_version 42920 (0.0007) -[2023-10-15 16:30:41,479][52833] Updated weights for policy 0, policy_version 42930 (0.0009) -[2023-10-15 16:30:41,853][52833] Updated weights for policy 0, policy_version 42940 (0.0010) -[2023-10-15 16:30:42,530][52866] Updated weights for policy 1, policy_version 43050 (0.0008) -[2023-10-15 16:30:42,894][52866] Updated weights for policy 1, policy_version 43060 (0.0008) -[2023-10-15 16:30:43,269][52866] Updated weights for policy 1, policy_version 43070 (0.0008) -[2023-10-15 16:30:43,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 88080384. Throughput: 0: 1806.7, 1: 1811.1. Samples: 22025740. Policy #0 lag: (min: 19.0, avg: 19.0, max: 22.0) -[2023-10-15 16:30:43,442][51532] Avg episode reward: [(0, '45.500'), (1, '45.820')] -[2023-10-15 16:30:45,567][52833] Updated weights for policy 0, policy_version 42950 (0.0008) -[2023-10-15 16:30:45,934][52833] Updated weights for policy 0, policy_version 42960 (0.0009) -[2023-10-15 16:30:46,309][52833] Updated weights for policy 0, policy_version 42970 (0.0009) -[2023-10-15 16:30:47,199][52866] Updated weights for policy 1, policy_version 43080 (0.0008) -[2023-10-15 16:30:47,563][52866] Updated weights for policy 1, policy_version 43090 (0.0009) -[2023-10-15 16:30:47,938][52866] Updated weights for policy 1, policy_version 43100 (0.0009) -[2023-10-15 16:30:48,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 88145920. Throughput: 0: 1815.2, 1: 1788.0. Samples: 22037064. Policy #0 lag: (min: 19.0, avg: 19.0, max: 22.0) -[2023-10-15 16:30:48,441][51532] Avg episode reward: [(0, '45.390'), (1, '44.380')] -[2023-10-15 16:30:50,098][52833] Updated weights for policy 0, policy_version 42980 (0.0008) -[2023-10-15 16:30:50,465][52833] Updated weights for policy 0, policy_version 42990 (0.0007) -[2023-10-15 16:30:50,846][52833] Updated weights for policy 0, policy_version 43000 (0.0008) -[2023-10-15 16:30:51,643][52866] Updated weights for policy 1, policy_version 43110 (0.0007) -[2023-10-15 16:30:52,012][52866] Updated weights for policy 1, policy_version 43120 (0.0007) -[2023-10-15 16:30:52,380][52866] Updated weights for policy 1, policy_version 43130 (0.0009) -[2023-10-15 16:30:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 88211456. Throughput: 0: 1803.4, 1: 1805.5. Samples: 22058188. Policy #0 lag: (min: 19.0, avg: 19.0, max: 22.0) -[2023-10-15 16:30:53,442][51532] Avg episode reward: [(0, '43.580'), (1, '46.180')] -[2023-10-15 16:30:54,598][52833] Updated weights for policy 0, policy_version 43010 (0.0009) -[2023-10-15 16:30:54,959][52833] Updated weights for policy 0, policy_version 43020 (0.0010) -[2023-10-15 16:30:55,333][52833] Updated weights for policy 0, policy_version 43030 (0.0010) -[2023-10-15 16:30:55,703][52833] Updated weights for policy 0, policy_version 43040 (0.0012) -[2023-10-15 16:30:56,124][52866] Updated weights for policy 1, policy_version 43140 (0.0009) -[2023-10-15 16:30:56,505][52866] Updated weights for policy 1, policy_version 43150 (0.0009) -[2023-10-15 16:30:56,883][52866] Updated weights for policy 1, policy_version 43160 (0.0009) -[2023-10-15 16:30:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 88276992. Throughput: 0: 1802.4, 1: 1791.9. Samples: 22079866. Policy #0 lag: (min: 19.0, avg: 19.0, max: 22.0) -[2023-10-15 16:30:58,442][51532] Avg episode reward: [(0, '43.410'), (1, '47.170')] -[2023-10-15 16:30:59,531][52833] Updated weights for policy 0, policy_version 43050 (0.0010) -[2023-10-15 16:30:59,905][52833] Updated weights for policy 0, policy_version 43060 (0.0009) -[2023-10-15 16:31:00,259][52833] Updated weights for policy 0, policy_version 43070 (0.0007) -[2023-10-15 16:31:00,484][52866] Updated weights for policy 1, policy_version 43170 (0.0009) -[2023-10-15 16:31:00,852][52866] Updated weights for policy 1, policy_version 43180 (0.0007) -[2023-10-15 16:31:01,207][52866] Updated weights for policy 1, policy_version 43190 (0.0009) -[2023-10-15 16:31:01,575][52866] Updated weights for policy 1, policy_version 43200 (0.0011) -[2023-10-15 16:31:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 88342528. Throughput: 0: 1803.3, 1: 1806.8. Samples: 22090586. Policy #0 lag: (min: 19.0, avg: 19.0, max: 22.0) -[2023-10-15 16:31:03,442][51532] Avg episode reward: [(0, '43.750'), (1, '46.760')] -[2023-10-15 16:31:04,209][52833] Updated weights for policy 0, policy_version 43080 (0.0009) -[2023-10-15 16:31:04,590][52833] Updated weights for policy 0, policy_version 43090 (0.0009) -[2023-10-15 16:31:04,948][52833] Updated weights for policy 0, policy_version 43100 (0.0007) -[2023-10-15 16:31:05,369][52866] Updated weights for policy 1, policy_version 43210 (0.0007) -[2023-10-15 16:31:05,737][52866] Updated weights for policy 1, policy_version 43220 (0.0008) -[2023-10-15 16:31:06,100][52866] Updated weights for policy 1, policy_version 43230 (0.0009) -[2023-10-15 16:31:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 88408064. Throughput: 0: 1802.5, 1: 1796.5. Samples: 22112160. Policy #0 lag: (min: 19.0, avg: 19.0, max: 22.0) -[2023-10-15 16:31:08,441][51532] Avg episode reward: [(0, '44.060'), (1, '50.260')] -[2023-10-15 16:31:08,442][52518] Saving new best policy, reward=50.260! -[2023-10-15 16:31:08,638][52833] Updated weights for policy 0, policy_version 43110 (0.0008) -[2023-10-15 16:31:09,010][52833] Updated weights for policy 0, policy_version 43120 (0.0009) -[2023-10-15 16:31:09,382][52833] Updated weights for policy 0, policy_version 43130 (0.0008) -[2023-10-15 16:31:09,814][52866] Updated weights for policy 1, policy_version 43240 (0.0009) -[2023-10-15 16:31:10,187][52866] Updated weights for policy 1, policy_version 43250 (0.0011) -[2023-10-15 16:31:10,548][52866] Updated weights for policy 1, policy_version 43260 (0.0007) -[2023-10-15 16:31:13,090][52833] Updated weights for policy 0, policy_version 43140 (0.0008) -[2023-10-15 16:31:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 88473600. Throughput: 0: 1816.7, 1: 1799.6. Samples: 22134966. Policy #0 lag: (min: 19.0, avg: 19.0, max: 22.0) -[2023-10-15 16:31:13,441][51532] Avg episode reward: [(0, '42.420'), (1, '48.360')] -[2023-10-15 16:31:13,459][52833] Updated weights for policy 0, policy_version 43150 (0.0007) -[2023-10-15 16:31:13,831][52833] Updated weights for policy 0, policy_version 43160 (0.0009) -[2023-10-15 16:31:14,208][52866] Updated weights for policy 1, policy_version 43270 (0.0008) -[2023-10-15 16:31:14,576][52866] Updated weights for policy 1, policy_version 43280 (0.0009) -[2023-10-15 16:31:14,946][52866] Updated weights for policy 1, policy_version 43290 (0.0009) -[2023-10-15 16:31:17,505][52833] Updated weights for policy 0, policy_version 43170 (0.0010) -[2023-10-15 16:31:17,870][52833] Updated weights for policy 0, policy_version 43180 (0.0009) -[2023-10-15 16:31:18,246][52833] Updated weights for policy 0, policy_version 43190 (0.0010) -[2023-10-15 16:31:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 88539136. Throughput: 0: 1798.1, 1: 1806.1. Samples: 22145144. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-15 16:31:18,441][51532] Avg episode reward: [(0, '43.550'), (1, '48.580')] -[2023-10-15 16:31:18,610][52833] Updated weights for policy 0, policy_version 43200 (0.0010) -[2023-10-15 16:31:18,672][52866] Updated weights for policy 1, policy_version 43300 (0.0008) -[2023-10-15 16:31:19,042][52866] Updated weights for policy 1, policy_version 43310 (0.0007) -[2023-10-15 16:31:19,417][52866] Updated weights for policy 1, policy_version 43320 (0.0011) -[2023-10-15 16:31:22,434][52833] Updated weights for policy 0, policy_version 43210 (0.0008) -[2023-10-15 16:31:22,802][52833] Updated weights for policy 0, policy_version 43220 (0.0007) -[2023-10-15 16:31:23,138][52866] Updated weights for policy 1, policy_version 43330 (0.0009) -[2023-10-15 16:31:23,175][52833] Updated weights for policy 0, policy_version 43230 (0.0007) -[2023-10-15 16:31:23,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 88637440. Throughput: 0: 1806.5, 1: 1806.8. Samples: 22167314. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-15 16:31:23,442][51532] Avg episode reward: [(0, '44.710'), (1, '49.640')] -[2023-10-15 16:31:23,515][52866] Updated weights for policy 1, policy_version 43340 (0.0008) -[2023-10-15 16:31:23,889][52866] Updated weights for policy 1, policy_version 43350 (0.0008) -[2023-10-15 16:31:24,250][52866] Updated weights for policy 1, policy_version 43360 (0.0007) -[2023-10-15 16:31:26,796][52833] Updated weights for policy 0, policy_version 43240 (0.0007) -[2023-10-15 16:31:27,167][52833] Updated weights for policy 0, policy_version 43250 (0.0008) -[2023-10-15 16:31:27,531][52833] Updated weights for policy 0, policy_version 43260 (0.0010) -[2023-10-15 16:31:28,089][52866] Updated weights for policy 1, policy_version 43370 (0.0008) -[2023-10-15 16:31:28,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 88702976. Throughput: 0: 1786.5, 1: 1818.5. Samples: 22187968. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-15 16:31:28,442][51532] Avg episode reward: [(0, '44.680'), (1, '49.280')] -[2023-10-15 16:31:28,455][52866] Updated weights for policy 1, policy_version 43380 (0.0008) -[2023-10-15 16:31:28,823][52866] Updated weights for policy 1, policy_version 43390 (0.0010) -[2023-10-15 16:31:31,276][52833] Updated weights for policy 0, policy_version 43270 (0.0009) -[2023-10-15 16:31:31,652][52833] Updated weights for policy 0, policy_version 43280 (0.0007) -[2023-10-15 16:31:32,015][52833] Updated weights for policy 0, policy_version 43290 (0.0007) -[2023-10-15 16:31:32,387][52866] Updated weights for policy 1, policy_version 43400 (0.0007) -[2023-10-15 16:31:32,748][52866] Updated weights for policy 1, policy_version 43410 (0.0009) -[2023-10-15 16:31:33,112][52866] Updated weights for policy 1, policy_version 43420 (0.0010) -[2023-10-15 16:31:33,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 88801280. Throughput: 0: 1803.8, 1: 1812.4. Samples: 22199796. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-15 16:31:33,442][51532] Avg episode reward: [(0, '47.780'), (1, '51.630')] -[2023-10-15 16:31:33,443][52518] Saving new best policy, reward=51.630! -[2023-10-15 16:31:35,747][52833] Updated weights for policy 0, policy_version 43300 (0.0007) -[2023-10-15 16:31:36,126][52833] Updated weights for policy 0, policy_version 43310 (0.0008) -[2023-10-15 16:31:36,492][52833] Updated weights for policy 0, policy_version 43320 (0.0009) -[2023-10-15 16:31:36,899][52866] Updated weights for policy 1, policy_version 43430 (0.0008) -[2023-10-15 16:31:37,268][52866] Updated weights for policy 1, policy_version 43440 (0.0009) -[2023-10-15 16:31:37,633][52866] Updated weights for policy 1, policy_version 43450 (0.0007) -[2023-10-15 16:31:38,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 88866816. Throughput: 0: 1786.9, 1: 1815.6. Samples: 22220300. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-15 16:31:38,442][51532] Avg episode reward: [(0, '45.650'), (1, '50.180')] -[2023-10-15 16:31:40,029][52833] Updated weights for policy 0, policy_version 43330 (0.0010) -[2023-10-15 16:31:40,399][52833] Updated weights for policy 0, policy_version 43340 (0.0011) -[2023-10-15 16:31:40,769][52833] Updated weights for policy 0, policy_version 43350 (0.0008) -[2023-10-15 16:31:41,129][52833] Updated weights for policy 0, policy_version 43360 (0.0008) -[2023-10-15 16:31:41,222][52866] Updated weights for policy 1, policy_version 43460 (0.0008) -[2023-10-15 16:31:41,601][52866] Updated weights for policy 1, policy_version 43470 (0.0009) -[2023-10-15 16:31:41,973][52866] Updated weights for policy 1, policy_version 43480 (0.0011) -[2023-10-15 16:31:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 88932352. Throughput: 0: 1799.9, 1: 1809.2. Samples: 22242272. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-15 16:31:43,442][51532] Avg episode reward: [(0, '43.690'), (1, '50.050')] -[2023-10-15 16:31:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000043488_44531712.pth... -[2023-10-15 16:31:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000043360_44400640.pth... -[2023-10-15 16:31:43,495][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000041792_42795008.pth -[2023-10-15 16:31:43,495][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000041696_42696704.pth -[2023-10-15 16:31:44,732][52833] Updated weights for policy 0, policy_version 43370 (0.0007) -[2023-10-15 16:31:45,105][52833] Updated weights for policy 0, policy_version 43380 (0.0008) -[2023-10-15 16:31:45,471][52833] Updated weights for policy 0, policy_version 43390 (0.0009) -[2023-10-15 16:31:45,720][52866] Updated weights for policy 1, policy_version 43490 (0.0009) -[2023-10-15 16:31:46,088][52866] Updated weights for policy 1, policy_version 43500 (0.0007) -[2023-10-15 16:31:46,460][52866] Updated weights for policy 1, policy_version 43510 (0.0010) -[2023-10-15 16:31:46,820][52866] Updated weights for policy 1, policy_version 43520 (0.0010) -[2023-10-15 16:31:48,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 88997888. Throughput: 0: 1798.6, 1: 1814.3. Samples: 22253164. Policy #0 lag: (min: 31.0, avg: 50.5, max: 63.0) -[2023-10-15 16:31:48,442][51532] Avg episode reward: [(0, '41.150'), (1, '49.580')] -[2023-10-15 16:31:49,272][52833] Updated weights for policy 0, policy_version 43400 (0.0007) -[2023-10-15 16:31:49,638][52833] Updated weights for policy 0, policy_version 43410 (0.0009) -[2023-10-15 16:31:50,012][52833] Updated weights for policy 0, policy_version 43420 (0.0010) -[2023-10-15 16:31:50,517][52866] Updated weights for policy 1, policy_version 43530 (0.0009) -[2023-10-15 16:31:50,884][52866] Updated weights for policy 1, policy_version 43540 (0.0007) -[2023-10-15 16:31:51,255][52866] Updated weights for policy 1, policy_version 43550 (0.0007) -[2023-10-15 16:31:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 89063424. Throughput: 0: 1803.3, 1: 1807.0. Samples: 22274622. Policy #0 lag: (min: 31.0, avg: 50.5, max: 63.0) -[2023-10-15 16:31:53,442][51532] Avg episode reward: [(0, '42.740'), (1, '49.200')] -[2023-10-15 16:31:53,748][52833] Updated weights for policy 0, policy_version 43430 (0.0010) -[2023-10-15 16:31:54,122][52833] Updated weights for policy 0, policy_version 43440 (0.0009) -[2023-10-15 16:31:54,491][52833] Updated weights for policy 0, policy_version 43450 (0.0011) -[2023-10-15 16:31:54,936][52866] Updated weights for policy 1, policy_version 43560 (0.0008) -[2023-10-15 16:31:55,300][52866] Updated weights for policy 1, policy_version 43570 (0.0007) -[2023-10-15 16:31:55,668][52866] Updated weights for policy 1, policy_version 43580 (0.0008) -[2023-10-15 16:31:58,215][52833] Updated weights for policy 0, policy_version 43460 (0.0010) -[2023-10-15 16:31:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 89128960. Throughput: 0: 1802.9, 1: 1800.7. Samples: 22297130. Policy #0 lag: (min: 31.0, avg: 50.5, max: 63.0) -[2023-10-15 16:31:58,442][51532] Avg episode reward: [(0, '42.280'), (1, '46.920')] -[2023-10-15 16:31:58,591][52833] Updated weights for policy 0, policy_version 43470 (0.0009) -[2023-10-15 16:31:58,961][52833] Updated weights for policy 0, policy_version 43480 (0.0008) -[2023-10-15 16:31:59,470][52866] Updated weights for policy 1, policy_version 43590 (0.0009) -[2023-10-15 16:31:59,831][52866] Updated weights for policy 1, policy_version 43600 (0.0008) -[2023-10-15 16:32:00,217][52866] Updated weights for policy 1, policy_version 43610 (0.0009) -[2023-10-15 16:32:02,759][52833] Updated weights for policy 0, policy_version 43490 (0.0008) -[2023-10-15 16:32:03,134][52833] Updated weights for policy 0, policy_version 43500 (0.0008) -[2023-10-15 16:32:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 89194496. Throughput: 0: 1801.4, 1: 1796.9. Samples: 22307068. Policy #0 lag: (min: 31.0, avg: 50.5, max: 63.0) -[2023-10-15 16:32:03,441][51532] Avg episode reward: [(0, '43.310'), (1, '46.170')] -[2023-10-15 16:32:03,502][52833] Updated weights for policy 0, policy_version 43510 (0.0008) -[2023-10-15 16:32:03,874][52833] Updated weights for policy 0, policy_version 43520 (0.0008) -[2023-10-15 16:32:04,005][52866] Updated weights for policy 1, policy_version 43620 (0.0007) -[2023-10-15 16:32:04,375][52866] Updated weights for policy 1, policy_version 43630 (0.0008) -[2023-10-15 16:32:04,733][52866] Updated weights for policy 1, policy_version 43640 (0.0008) -[2023-10-15 16:32:07,830][52833] Updated weights for policy 0, policy_version 43530 (0.0008) -[2023-10-15 16:32:08,204][52833] Updated weights for policy 0, policy_version 43540 (0.0008) -[2023-10-15 16:32:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 89260032. Throughput: 0: 1800.9, 1: 1794.7. Samples: 22329118. Policy #0 lag: (min: 31.0, avg: 50.5, max: 63.0) -[2023-10-15 16:32:08,442][51532] Avg episode reward: [(0, '44.630'), (1, '44.330')] -[2023-10-15 16:32:08,478][52866] Updated weights for policy 1, policy_version 43650 (0.0007) -[2023-10-15 16:32:08,573][52833] Updated weights for policy 0, policy_version 43550 (0.0008) -[2023-10-15 16:32:08,841][52866] Updated weights for policy 1, policy_version 43660 (0.0007) -[2023-10-15 16:32:09,204][52866] Updated weights for policy 1, policy_version 43670 (0.0009) -[2023-10-15 16:32:09,577][52866] Updated weights for policy 1, policy_version 43680 (0.0010) -[2023-10-15 16:32:12,216][52833] Updated weights for policy 0, policy_version 43560 (0.0008) -[2023-10-15 16:32:12,589][52833] Updated weights for policy 0, policy_version 43570 (0.0007) -[2023-10-15 16:32:12,957][52833] Updated weights for policy 0, policy_version 43580 (0.0008) -[2023-10-15 16:32:13,352][52866] Updated weights for policy 1, policy_version 43690 (0.0010) -[2023-10-15 16:32:13,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 89358336. Throughput: 0: 1810.4, 1: 1810.3. Samples: 22350898. Policy #0 lag: (min: 31.0, avg: 50.5, max: 63.0) -[2023-10-15 16:32:13,442][51532] Avg episode reward: [(0, '37.770'), (1, '44.660')] -[2023-10-15 16:32:13,722][52866] Updated weights for policy 1, policy_version 43700 (0.0009) -[2023-10-15 16:32:14,087][52866] Updated weights for policy 1, policy_version 43710 (0.0010) -[2023-10-15 16:32:16,703][52833] Updated weights for policy 0, policy_version 43590 (0.0010) -[2023-10-15 16:32:17,065][52833] Updated weights for policy 0, policy_version 43600 (0.0007) -[2023-10-15 16:32:17,433][52833] Updated weights for policy 0, policy_version 43610 (0.0008) -[2023-10-15 16:32:17,880][52866] Updated weights for policy 1, policy_version 43720 (0.0009) -[2023-10-15 16:32:18,251][52866] Updated weights for policy 1, policy_version 43730 (0.0009) -[2023-10-15 16:32:18,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 89423872. Throughput: 0: 1801.0, 1: 1799.8. Samples: 22361834. Policy #0 lag: (min: 27.0, avg: 27.7, max: 45.0) -[2023-10-15 16:32:18,441][51532] Avg episode reward: [(0, '39.240'), (1, '43.430')] -[2023-10-15 16:32:18,616][52866] Updated weights for policy 1, policy_version 43740 (0.0008) -[2023-10-15 16:32:21,086][52833] Updated weights for policy 0, policy_version 43620 (0.0008) -[2023-10-15 16:32:21,464][52833] Updated weights for policy 0, policy_version 43630 (0.0010) -[2023-10-15 16:32:21,833][52833] Updated weights for policy 0, policy_version 43640 (0.0010) -[2023-10-15 16:32:22,353][52866] Updated weights for policy 1, policy_version 43750 (0.0007) -[2023-10-15 16:32:22,712][52866] Updated weights for policy 1, policy_version 43760 (0.0008) -[2023-10-15 16:32:23,086][52866] Updated weights for policy 1, policy_version 43770 (0.0009) -[2023-10-15 16:32:23,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 89522176. Throughput: 0: 1812.4, 1: 1815.2. Samples: 22383542. Policy #0 lag: (min: 27.0, avg: 27.7, max: 45.0) -[2023-10-15 16:32:23,442][51532] Avg episode reward: [(0, '41.470'), (1, '42.520')] -[2023-10-15 16:32:25,565][52833] Updated weights for policy 0, policy_version 43650 (0.0009) -[2023-10-15 16:32:25,927][52833] Updated weights for policy 0, policy_version 43660 (0.0009) -[2023-10-15 16:32:26,295][52833] Updated weights for policy 0, policy_version 43670 (0.0011) -[2023-10-15 16:32:26,669][52833] Updated weights for policy 0, policy_version 43680 (0.0008) -[2023-10-15 16:32:26,916][52866] Updated weights for policy 1, policy_version 43780 (0.0010) -[2023-10-15 16:32:27,294][52866] Updated weights for policy 1, policy_version 43790 (0.0010) -[2023-10-15 16:32:27,654][52866] Updated weights for policy 1, policy_version 43800 (0.0010) -[2023-10-15 16:32:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 89587712. Throughput: 0: 1793.4, 1: 1803.9. Samples: 22404150. Policy #0 lag: (min: 27.0, avg: 27.7, max: 45.0) -[2023-10-15 16:32:28,441][51532] Avg episode reward: [(0, '43.870'), (1, '40.560')] -[2023-10-15 16:32:30,319][52833] Updated weights for policy 0, policy_version 43690 (0.0010) -[2023-10-15 16:32:30,680][52833] Updated weights for policy 0, policy_version 43700 (0.0008) -[2023-10-15 16:32:31,053][52833] Updated weights for policy 0, policy_version 43710 (0.0009) -[2023-10-15 16:32:31,436][52866] Updated weights for policy 1, policy_version 43810 (0.0009) -[2023-10-15 16:32:31,793][52866] Updated weights for policy 1, policy_version 43820 (0.0012) -[2023-10-15 16:32:32,162][52866] Updated weights for policy 1, policy_version 43830 (0.0008) -[2023-10-15 16:32:32,526][52866] Updated weights for policy 1, policy_version 43840 (0.0008) -[2023-10-15 16:32:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 89653248. Throughput: 0: 1803.9, 1: 1810.3. Samples: 22415802. Policy #0 lag: (min: 27.0, avg: 27.7, max: 45.0) -[2023-10-15 16:32:33,442][51532] Avg episode reward: [(0, '46.810'), (1, '39.740')] -[2023-10-15 16:32:34,740][52833] Updated weights for policy 0, policy_version 43720 (0.0009) -[2023-10-15 16:32:35,104][52833] Updated weights for policy 0, policy_version 43730 (0.0010) -[2023-10-15 16:32:35,472][52833] Updated weights for policy 0, policy_version 43740 (0.0009) -[2023-10-15 16:32:36,103][52866] Updated weights for policy 1, policy_version 43850 (0.0008) -[2023-10-15 16:32:36,478][52866] Updated weights for policy 1, policy_version 43860 (0.0009) -[2023-10-15 16:32:36,840][52866] Updated weights for policy 1, policy_version 43870 (0.0011) -[2023-10-15 16:32:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89718784. Throughput: 0: 1797.2, 1: 1800.5. Samples: 22436518. Policy #0 lag: (min: 27.0, avg: 27.7, max: 45.0) -[2023-10-15 16:32:38,442][51532] Avg episode reward: [(0, '46.640'), (1, '40.910')] -[2023-10-15 16:32:39,315][52833] Updated weights for policy 0, policy_version 43750 (0.0009) -[2023-10-15 16:32:39,697][52833] Updated weights for policy 0, policy_version 43760 (0.0007) -[2023-10-15 16:32:40,060][52833] Updated weights for policy 0, policy_version 43770 (0.0007) -[2023-10-15 16:32:40,547][52866] Updated weights for policy 1, policy_version 43880 (0.0008) -[2023-10-15 16:32:40,921][52866] Updated weights for policy 1, policy_version 43890 (0.0010) -[2023-10-15 16:32:41,279][52866] Updated weights for policy 1, policy_version 43900 (0.0010) -[2023-10-15 16:32:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89784320. Throughput: 0: 1793.7, 1: 1802.9. Samples: 22458978. Policy #0 lag: (min: 27.0, avg: 27.7, max: 45.0) -[2023-10-15 16:32:43,442][51532] Avg episode reward: [(0, '46.130'), (1, '42.180')] -[2023-10-15 16:32:43,697][52833] Updated weights for policy 0, policy_version 43780 (0.0008) -[2023-10-15 16:32:44,063][52833] Updated weights for policy 0, policy_version 43790 (0.0009) -[2023-10-15 16:32:44,435][52833] Updated weights for policy 0, policy_version 43800 (0.0009) -[2023-10-15 16:32:45,020][52866] Updated weights for policy 1, policy_version 43910 (0.0007) -[2023-10-15 16:32:45,395][52866] Updated weights for policy 1, policy_version 43920 (0.0008) -[2023-10-15 16:32:45,764][52866] Updated weights for policy 1, policy_version 43930 (0.0009) -[2023-10-15 16:32:48,114][52833] Updated weights for policy 0, policy_version 43810 (0.0010) -[2023-10-15 16:32:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89849856. Throughput: 0: 1794.0, 1: 1805.9. Samples: 22469062. Policy #0 lag: (min: 27.0, avg: 27.7, max: 45.0) -[2023-10-15 16:32:48,442][51532] Avg episode reward: [(0, '49.740'), (1, '42.310')] -[2023-10-15 16:32:48,480][52833] Updated weights for policy 0, policy_version 43820 (0.0009) -[2023-10-15 16:32:48,859][52833] Updated weights for policy 0, policy_version 43830 (0.0008) -[2023-10-15 16:32:49,221][52410] Saving new best policy, reward=49.740! -[2023-10-15 16:32:49,225][52833] Updated weights for policy 0, policy_version 43840 (0.0010) -[2023-10-15 16:32:49,584][52866] Updated weights for policy 1, policy_version 43940 (0.0009) -[2023-10-15 16:32:49,944][52866] Updated weights for policy 1, policy_version 43950 (0.0008) -[2023-10-15 16:32:50,307][52866] Updated weights for policy 1, policy_version 43960 (0.0009) -[2023-10-15 16:32:53,037][52833] Updated weights for policy 0, policy_version 43850 (0.0010) -[2023-10-15 16:32:53,404][52833] Updated weights for policy 0, policy_version 43860 (0.0008) -[2023-10-15 16:32:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 89915392. Throughput: 0: 1801.9, 1: 1802.3. Samples: 22491306. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:32:53,442][51532] Avg episode reward: [(0, '48.040'), (1, '42.710')] -[2023-10-15 16:32:53,771][52833] Updated weights for policy 0, policy_version 43870 (0.0010) -[2023-10-15 16:32:54,050][52866] Updated weights for policy 1, policy_version 43970 (0.0008) -[2023-10-15 16:32:54,411][52866] Updated weights for policy 1, policy_version 43980 (0.0008) -[2023-10-15 16:32:54,785][52866] Updated weights for policy 1, policy_version 43990 (0.0007) -[2023-10-15 16:32:55,134][52866] Updated weights for policy 1, policy_version 44000 (0.0007) -[2023-10-15 16:32:57,434][52833] Updated weights for policy 0, policy_version 43880 (0.0008) -[2023-10-15 16:32:57,800][52833] Updated weights for policy 0, policy_version 43890 (0.0009) -[2023-10-15 16:32:58,165][52833] Updated weights for policy 0, policy_version 43900 (0.0007) -[2023-10-15 16:32:58,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 90013696. Throughput: 0: 1804.6, 1: 1797.8. Samples: 22513004. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:32:58,442][51532] Avg episode reward: [(0, '48.480'), (1, '42.470')] -[2023-10-15 16:32:58,836][52866] Updated weights for policy 1, policy_version 44010 (0.0011) -[2023-10-15 16:32:59,193][52866] Updated weights for policy 1, policy_version 44020 (0.0011) -[2023-10-15 16:32:59,562][52866] Updated weights for policy 1, policy_version 44030 (0.0010) -[2023-10-15 16:33:02,007][52833] Updated weights for policy 0, policy_version 43910 (0.0010) -[2023-10-15 16:33:02,382][52833] Updated weights for policy 0, policy_version 43920 (0.0009) -[2023-10-15 16:33:02,761][52833] Updated weights for policy 0, policy_version 43930 (0.0009) -[2023-10-15 16:33:03,330][52866] Updated weights for policy 1, policy_version 44040 (0.0008) -[2023-10-15 16:33:03,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 90079232. Throughput: 0: 1798.6, 1: 1797.3. Samples: 22523648. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:33:03,441][51532] Avg episode reward: [(0, '50.220'), (1, '43.440')] -[2023-10-15 16:33:03,442][52410] Saving new best policy, reward=50.220! -[2023-10-15 16:33:03,701][52866] Updated weights for policy 1, policy_version 44050 (0.0009) -[2023-10-15 16:33:04,074][52866] Updated weights for policy 1, policy_version 44060 (0.0009) -[2023-10-15 16:33:06,586][52833] Updated weights for policy 0, policy_version 43940 (0.0009) -[2023-10-15 16:33:06,952][52833] Updated weights for policy 0, policy_version 43950 (0.0010) -[2023-10-15 16:33:07,316][52833] Updated weights for policy 0, policy_version 43960 (0.0011) -[2023-10-15 16:33:07,825][52866] Updated weights for policy 1, policy_version 44070 (0.0010) -[2023-10-15 16:33:08,200][52866] Updated weights for policy 1, policy_version 44080 (0.0009) -[2023-10-15 16:33:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 90144768. Throughput: 0: 1803.6, 1: 1791.7. Samples: 22545330. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:33:08,441][51532] Avg episode reward: [(0, '49.730'), (1, '44.610')] -[2023-10-15 16:33:08,576][52866] Updated weights for policy 1, policy_version 44090 (0.0009) -[2023-10-15 16:33:11,225][52833] Updated weights for policy 0, policy_version 43970 (0.0008) -[2023-10-15 16:33:11,587][52833] Updated weights for policy 0, policy_version 43980 (0.0008) -[2023-10-15 16:33:11,962][52833] Updated weights for policy 0, policy_version 43990 (0.0009) -[2023-10-15 16:33:12,312][52866] Updated weights for policy 1, policy_version 44100 (0.0009) -[2023-10-15 16:33:12,319][52833] Updated weights for policy 0, policy_version 44000 (0.0009) -[2023-10-15 16:33:12,699][52866] Updated weights for policy 1, policy_version 44110 (0.0011) -[2023-10-15 16:33:13,070][52866] Updated weights for policy 1, policy_version 44120 (0.0010) -[2023-10-15 16:33:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 90243072. Throughput: 0: 1785.9, 1: 1802.1. Samples: 22565608. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:33:13,441][51532] Avg episode reward: [(0, '45.940'), (1, '43.620')] -[2023-10-15 16:33:15,982][52833] Updated weights for policy 0, policy_version 44010 (0.0008) -[2023-10-15 16:33:16,344][52833] Updated weights for policy 0, policy_version 44020 (0.0007) -[2023-10-15 16:33:16,718][52833] Updated weights for policy 0, policy_version 44030 (0.0009) -[2023-10-15 16:33:16,721][52866] Updated weights for policy 1, policy_version 44130 (0.0009) -[2023-10-15 16:33:17,085][52866] Updated weights for policy 1, policy_version 44140 (0.0007) -[2023-10-15 16:33:17,448][52866] Updated weights for policy 1, policy_version 44150 (0.0008) -[2023-10-15 16:33:17,818][52866] Updated weights for policy 1, policy_version 44160 (0.0007) -[2023-10-15 16:33:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 90308608. Throughput: 0: 1801.5, 1: 1792.3. Samples: 22577520. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:33:18,442][51532] Avg episode reward: [(0, '44.490'), (1, '42.990')] -[2023-10-15 16:33:20,595][52833] Updated weights for policy 0, policy_version 44040 (0.0007) -[2023-10-15 16:33:20,963][52833] Updated weights for policy 0, policy_version 44050 (0.0008) -[2023-10-15 16:33:21,325][52833] Updated weights for policy 0, policy_version 44060 (0.0009) -[2023-10-15 16:33:21,663][52866] Updated weights for policy 1, policy_version 44170 (0.0009) -[2023-10-15 16:33:22,030][52866] Updated weights for policy 1, policy_version 44180 (0.0010) -[2023-10-15 16:33:22,395][52866] Updated weights for policy 1, policy_version 44190 (0.0009) -[2023-10-15 16:33:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90374144. Throughput: 0: 1779.0, 1: 1809.3. Samples: 22597994. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-15 16:33:23,442][51532] Avg episode reward: [(0, '45.630'), (1, '44.300')] -[2023-10-15 16:33:25,095][52833] Updated weights for policy 0, policy_version 44070 (0.0007) -[2023-10-15 16:33:25,480][52833] Updated weights for policy 0, policy_version 44080 (0.0008) -[2023-10-15 16:33:25,843][52833] Updated weights for policy 0, policy_version 44090 (0.0008) -[2023-10-15 16:33:26,091][52866] Updated weights for policy 1, policy_version 44200 (0.0008) -[2023-10-15 16:33:26,463][52866] Updated weights for policy 1, policy_version 44210 (0.0008) -[2023-10-15 16:33:26,825][52866] Updated weights for policy 1, policy_version 44220 (0.0008) -[2023-10-15 16:33:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 90439680. Throughput: 0: 1779.5, 1: 1794.9. Samples: 22619824. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-15 16:33:28,442][51532] Avg episode reward: [(0, '46.510'), (1, '45.280')] -[2023-10-15 16:33:29,682][52833] Updated weights for policy 0, policy_version 44100 (0.0008) -[2023-10-15 16:33:30,053][52833] Updated weights for policy 0, policy_version 44110 (0.0008) -[2023-10-15 16:33:30,426][52833] Updated weights for policy 0, policy_version 44120 (0.0009) -[2023-10-15 16:33:30,484][52866] Updated weights for policy 1, policy_version 44230 (0.0008) -[2023-10-15 16:33:30,855][52866] Updated weights for policy 1, policy_version 44240 (0.0008) -[2023-10-15 16:33:31,220][52866] Updated weights for policy 1, policy_version 44250 (0.0008) -[2023-10-15 16:33:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90505216. Throughput: 0: 1773.9, 1: 1803.8. Samples: 22630060. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-15 16:33:33,442][51532] Avg episode reward: [(0, '45.080'), (1, '46.130')] -[2023-10-15 16:33:34,181][52833] Updated weights for policy 0, policy_version 44130 (0.0007) -[2023-10-15 16:33:34,548][52833] Updated weights for policy 0, policy_version 44140 (0.0009) -[2023-10-15 16:33:34,889][52866] Updated weights for policy 1, policy_version 44260 (0.0008) -[2023-10-15 16:33:34,927][52833] Updated weights for policy 0, policy_version 44150 (0.0008) -[2023-10-15 16:33:35,256][52866] Updated weights for policy 1, policy_version 44270 (0.0008) -[2023-10-15 16:33:35,287][52833] Updated weights for policy 0, policy_version 44160 (0.0008) -[2023-10-15 16:33:35,628][52866] Updated weights for policy 1, policy_version 44280 (0.0008) -[2023-10-15 16:33:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90570752. Throughput: 0: 1774.0, 1: 1799.4. Samples: 22652108. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-15 16:33:38,441][51532] Avg episode reward: [(0, '44.830'), (1, '47.540')] -[2023-10-15 16:33:39,206][52833] Updated weights for policy 0, policy_version 44170 (0.0009) -[2023-10-15 16:33:39,314][52866] Updated weights for policy 1, policy_version 44290 (0.0007) -[2023-10-15 16:33:39,584][52833] Updated weights for policy 0, policy_version 44180 (0.0008) -[2023-10-15 16:33:39,678][52866] Updated weights for policy 1, policy_version 44300 (0.0008) -[2023-10-15 16:33:39,942][52833] Updated weights for policy 0, policy_version 44190 (0.0007) -[2023-10-15 16:33:40,056][52866] Updated weights for policy 1, policy_version 44310 (0.0008) -[2023-10-15 16:33:40,412][52866] Updated weights for policy 1, policy_version 44320 (0.0010) -[2023-10-15 16:33:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14329.1). Total num frames: 90636288. Throughput: 0: 1792.5, 1: 1798.4. Samples: 22674594. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-15 16:33:43,441][51532] Avg episode reward: [(0, '44.010'), (1, '48.960')] -[2023-10-15 16:33:43,449][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000044320_45383680.pth... -[2023-10-15 16:33:43,478][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000042656_43679744.pth -[2023-10-15 16:33:43,777][52833] Updated weights for policy 0, policy_version 44200 (0.0007) -[2023-10-15 16:33:44,099][52866] Updated weights for policy 1, policy_version 44330 (0.0007) -[2023-10-15 16:33:44,147][52833] Updated weights for policy 0, policy_version 44210 (0.0007) -[2023-10-15 16:33:44,467][52866] Updated weights for policy 1, policy_version 44340 (0.0009) -[2023-10-15 16:33:44,516][52833] Updated weights for policy 0, policy_version 44220 (0.0007) -[2023-10-15 16:33:44,658][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000044224_45285376.pth... -[2023-10-15 16:33:44,694][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000042528_43548672.pth -[2023-10-15 16:33:44,828][52866] Updated weights for policy 1, policy_version 44350 (0.0009) -[2023-10-15 16:33:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 90701824. Throughput: 0: 1772.6, 1: 1800.1. Samples: 22684418. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-15 16:33:48,442][51532] Avg episode reward: [(0, '46.480'), (1, '48.140')] -[2023-10-15 16:33:48,505][52833] Updated weights for policy 0, policy_version 44230 (0.0008) -[2023-10-15 16:33:48,627][52866] Updated weights for policy 1, policy_version 44360 (0.0009) -[2023-10-15 16:33:48,869][52833] Updated weights for policy 0, policy_version 44240 (0.0007) -[2023-10-15 16:33:48,990][52866] Updated weights for policy 1, policy_version 44370 (0.0008) -[2023-10-15 16:33:49,241][52833] Updated weights for policy 0, policy_version 44250 (0.0009) -[2023-10-15 16:33:49,362][52866] Updated weights for policy 1, policy_version 44380 (0.0007) -[2023-10-15 16:33:52,976][52833] Updated weights for policy 0, policy_version 44260 (0.0007) -[2023-10-15 16:33:53,132][52866] Updated weights for policy 1, policy_version 44390 (0.0008) -[2023-10-15 16:33:53,335][52833] Updated weights for policy 0, policy_version 44270 (0.0008) -[2023-10-15 16:33:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 90767360. Throughput: 0: 1776.5, 1: 1798.7. Samples: 22706214. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) -[2023-10-15 16:33:53,441][51532] Avg episode reward: [(0, '45.320'), (1, '49.580')] -[2023-10-15 16:33:53,491][52866] Updated weights for policy 1, policy_version 44400 (0.0008) -[2023-10-15 16:33:53,703][52833] Updated weights for policy 0, policy_version 44280 (0.0007) -[2023-10-15 16:33:53,854][52866] Updated weights for policy 1, policy_version 44410 (0.0008) -[2023-10-15 16:33:57,496][52833] Updated weights for policy 0, policy_version 44290 (0.0008) -[2023-10-15 16:33:57,632][52866] Updated weights for policy 1, policy_version 44420 (0.0008) -[2023-10-15 16:33:57,867][52833] Updated weights for policy 0, policy_version 44300 (0.0007) -[2023-10-15 16:33:58,011][52866] Updated weights for policy 1, policy_version 44430 (0.0007) -[2023-10-15 16:33:58,230][52833] Updated weights for policy 0, policy_version 44310 (0.0007) -[2023-10-15 16:33:58,377][52866] Updated weights for policy 1, policy_version 44440 (0.0009) -[2023-10-15 16:33:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 14218.0). Total num frames: 90832896. Throughput: 0: 1789.0, 1: 1810.0. Samples: 22727562. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) -[2023-10-15 16:33:58,441][51532] Avg episode reward: [(0, '45.280'), (1, '50.630')] -[2023-10-15 16:33:58,596][52833] Updated weights for policy 0, policy_version 44320 (0.0008) -[2023-10-15 16:34:02,037][52866] Updated weights for policy 1, policy_version 44450 (0.0008) -[2023-10-15 16:34:02,156][52833] Updated weights for policy 0, policy_version 44330 (0.0009) -[2023-10-15 16:34:02,393][52866] Updated weights for policy 1, policy_version 44460 (0.0007) -[2023-10-15 16:34:02,514][52833] Updated weights for policy 0, policy_version 44340 (0.0008) -[2023-10-15 16:34:02,760][52866] Updated weights for policy 1, policy_version 44470 (0.0008) -[2023-10-15 16:34:02,884][52833] Updated weights for policy 0, policy_version 44350 (0.0010) -[2023-10-15 16:34:03,125][52866] Updated weights for policy 1, policy_version 44480 (0.0007) -[2023-10-15 16:34:03,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 90963968. Throughput: 0: 1776.8, 1: 1799.2. Samples: 22738440. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) -[2023-10-15 16:34:03,441][51532] Avg episode reward: [(0, '44.890'), (1, '51.800')] -[2023-10-15 16:34:03,442][52518] Saving new best policy, reward=51.800! -[2023-10-15 16:34:06,619][52833] Updated weights for policy 0, policy_version 44360 (0.0009) -[2023-10-15 16:34:06,868][52866] Updated weights for policy 1, policy_version 44490 (0.0009) -[2023-10-15 16:34:06,989][52833] Updated weights for policy 0, policy_version 44370 (0.0008) -[2023-10-15 16:34:07,228][52866] Updated weights for policy 1, policy_version 44500 (0.0008) -[2023-10-15 16:34:07,367][52833] Updated weights for policy 0, policy_version 44380 (0.0008) -[2023-10-15 16:34:07,595][52866] Updated weights for policy 1, policy_version 44510 (0.0009) -[2023-10-15 16:34:08,441][51532] Fps is (10 sec: 19661.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 91029504. Throughput: 0: 1793.7, 1: 1804.4. Samples: 22759910. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) -[2023-10-15 16:34:08,441][51532] Avg episode reward: [(0, '43.740'), (1, '50.860')] -[2023-10-15 16:34:11,210][52833] Updated weights for policy 0, policy_version 44390 (0.0011) -[2023-10-15 16:34:11,302][52866] Updated weights for policy 1, policy_version 44520 (0.0008) -[2023-10-15 16:34:11,586][52833] Updated weights for policy 0, policy_version 44400 (0.0009) -[2023-10-15 16:34:11,665][52866] Updated weights for policy 1, policy_version 44530 (0.0007) -[2023-10-15 16:34:11,956][52833] Updated weights for policy 0, policy_version 44410 (0.0010) -[2023-10-15 16:34:12,026][52866] Updated weights for policy 1, policy_version 44540 (0.0007) -[2023-10-15 16:34:13,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 91095040. Throughput: 0: 1776.4, 1: 1796.5. Samples: 22780602. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) -[2023-10-15 16:34:13,442][51532] Avg episode reward: [(0, '44.040'), (1, '52.100')] -[2023-10-15 16:34:13,452][52518] Saving new best policy, reward=52.100! -[2023-10-15 16:34:15,723][52833] Updated weights for policy 0, policy_version 44420 (0.0008) -[2023-10-15 16:34:15,915][52866] Updated weights for policy 1, policy_version 44550 (0.0008) -[2023-10-15 16:34:16,081][52833] Updated weights for policy 0, policy_version 44430 (0.0007) -[2023-10-15 16:34:16,285][52866] Updated weights for policy 1, policy_version 44560 (0.0008) -[2023-10-15 16:34:16,462][52833] Updated weights for policy 0, policy_version 44440 (0.0007) -[2023-10-15 16:34:16,646][52866] Updated weights for policy 1, policy_version 44570 (0.0008) -[2023-10-15 16:34:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91160576. Throughput: 0: 1803.8, 1: 1809.6. Samples: 22792662. Policy #0 lag: (min: 24.0, avg: 39.1, max: 40.0) -[2023-10-15 16:34:18,442][51532] Avg episode reward: [(0, '43.480'), (1, '53.490')] -[2023-10-15 16:34:18,442][52518] Saving new best policy, reward=53.490! -[2023-10-15 16:34:20,194][52833] Updated weights for policy 0, policy_version 44450 (0.0009) -[2023-10-15 16:34:20,326][52866] Updated weights for policy 1, policy_version 44580 (0.0009) -[2023-10-15 16:34:20,558][52833] Updated weights for policy 0, policy_version 44460 (0.0007) -[2023-10-15 16:34:20,684][52866] Updated weights for policy 1, policy_version 44590 (0.0007) -[2023-10-15 16:34:20,933][52833] Updated weights for policy 0, policy_version 44470 (0.0007) -[2023-10-15 16:34:21,055][52866] Updated weights for policy 1, policy_version 44600 (0.0007) -[2023-10-15 16:34:21,297][52833] Updated weights for policy 0, policy_version 44480 (0.0009) -[2023-10-15 16:34:23,441][51532] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 91226112. Throughput: 0: 1778.4, 1: 1792.2. Samples: 22812788. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-15 16:34:23,441][51532] Avg episode reward: [(0, '44.190'), (1, '47.970')] -[2023-10-15 16:34:25,068][52833] Updated weights for policy 0, policy_version 44490 (0.0009) -[2023-10-15 16:34:25,076][52866] Updated weights for policy 1, policy_version 44610 (0.0011) -[2023-10-15 16:34:25,434][52866] Updated weights for policy 1, policy_version 44620 (0.0008) -[2023-10-15 16:34:25,437][52833] Updated weights for policy 0, policy_version 44500 (0.0007) -[2023-10-15 16:34:25,800][52866] Updated weights for policy 1, policy_version 44630 (0.0008) -[2023-10-15 16:34:25,810][52833] Updated weights for policy 0, policy_version 44510 (0.0007) -[2023-10-15 16:34:26,167][52866] Updated weights for policy 1, policy_version 44640 (0.0008) -[2023-10-15 16:34:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91291648. Throughput: 0: 1782.3, 1: 1792.9. Samples: 22835482. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-15 16:34:28,442][51532] Avg episode reward: [(0, '45.830'), (1, '45.000')] -[2023-10-15 16:34:29,632][52833] Updated weights for policy 0, policy_version 44520 (0.0008) -[2023-10-15 16:34:29,984][52866] Updated weights for policy 1, policy_version 44650 (0.0008) -[2023-10-15 16:34:30,003][52833] Updated weights for policy 0, policy_version 44530 (0.0007) -[2023-10-15 16:34:30,344][52866] Updated weights for policy 1, policy_version 44660 (0.0007) -[2023-10-15 16:34:30,369][52833] Updated weights for policy 0, policy_version 44540 (0.0007) -[2023-10-15 16:34:30,717][52866] Updated weights for policy 1, policy_version 44670 (0.0007) -[2023-10-15 16:34:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 91357184. Throughput: 0: 1780.7, 1: 1789.4. Samples: 22845072. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-15 16:34:33,441][51532] Avg episode reward: [(0, '44.600'), (1, '45.940')] -[2023-10-15 16:34:34,095][52833] Updated weights for policy 0, policy_version 44550 (0.0007) -[2023-10-15 16:34:34,462][52833] Updated weights for policy 0, policy_version 44560 (0.0009) -[2023-10-15 16:34:34,485][52866] Updated weights for policy 1, policy_version 44680 (0.0007) -[2023-10-15 16:34:34,835][52833] Updated weights for policy 0, policy_version 44570 (0.0008) -[2023-10-15 16:34:34,854][52866] Updated weights for policy 1, policy_version 44690 (0.0008) -[2023-10-15 16:34:35,221][52866] Updated weights for policy 1, policy_version 44700 (0.0008) -[2023-10-15 16:34:38,357][52833] Updated weights for policy 0, policy_version 44580 (0.0007) -[2023-10-15 16:34:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 91422720. Throughput: 0: 1795.3, 1: 1790.1. Samples: 22867560. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-15 16:34:38,441][51532] Avg episode reward: [(0, '45.680'), (1, '42.780')] -[2023-10-15 16:34:38,732][52833] Updated weights for policy 0, policy_version 44590 (0.0008) -[2023-10-15 16:34:38,800][52866] Updated weights for policy 1, policy_version 44710 (0.0008) -[2023-10-15 16:34:39,096][52833] Updated weights for policy 0, policy_version 44600 (0.0007) -[2023-10-15 16:34:39,166][52866] Updated weights for policy 1, policy_version 44720 (0.0008) -[2023-10-15 16:34:39,529][52866] Updated weights for policy 1, policy_version 44730 (0.0007) -[2023-10-15 16:34:42,885][52833] Updated weights for policy 0, policy_version 44610 (0.0008) -[2023-10-15 16:34:43,251][52833] Updated weights for policy 0, policy_version 44620 (0.0007) -[2023-10-15 16:34:43,358][52866] Updated weights for policy 1, policy_version 44740 (0.0009) -[2023-10-15 16:34:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 91488256. Throughput: 0: 1811.4, 1: 1802.2. Samples: 22890174. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-15 16:34:43,442][51532] Avg episode reward: [(0, '45.260'), (1, '42.440')] -[2023-10-15 16:34:43,611][52833] Updated weights for policy 0, policy_version 44630 (0.0008) -[2023-10-15 16:34:43,742][52866] Updated weights for policy 1, policy_version 44750 (0.0009) -[2023-10-15 16:34:43,975][52833] Updated weights for policy 0, policy_version 44640 (0.0011) -[2023-10-15 16:34:44,107][52866] Updated weights for policy 1, policy_version 44760 (0.0008) -[2023-10-15 16:34:47,759][52833] Updated weights for policy 0, policy_version 44650 (0.0007) -[2023-10-15 16:34:47,790][52866] Updated weights for policy 1, policy_version 44770 (0.0009) -[2023-10-15 16:34:48,122][52833] Updated weights for policy 0, policy_version 44660 (0.0008) -[2023-10-15 16:34:48,150][52866] Updated weights for policy 1, policy_version 44780 (0.0008) -[2023-10-15 16:34:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 91553792. Throughput: 0: 1797.7, 1: 1788.6. Samples: 22899826. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-15 16:34:48,442][51532] Avg episode reward: [(0, '46.250'), (1, '40.960')] -[2023-10-15 16:34:48,486][52833] Updated weights for policy 0, policy_version 44670 (0.0008) -[2023-10-15 16:34:48,511][52866] Updated weights for policy 1, policy_version 44790 (0.0010) -[2023-10-15 16:34:48,876][52866] Updated weights for policy 1, policy_version 44800 (0.0009) -[2023-10-15 16:34:52,308][52833] Updated weights for policy 0, policy_version 44680 (0.0009) -[2023-10-15 16:34:52,599][52866] Updated weights for policy 1, policy_version 44810 (0.0007) -[2023-10-15 16:34:52,674][52833] Updated weights for policy 0, policy_version 44690 (0.0009) -[2023-10-15 16:34:52,963][52866] Updated weights for policy 1, policy_version 44820 (0.0010) -[2023-10-15 16:34:53,053][52833] Updated weights for policy 0, policy_version 44700 (0.0007) -[2023-10-15 16:34:53,325][52866] Updated weights for policy 1, policy_version 44830 (0.0007) -[2023-10-15 16:34:53,441][51532] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 91684864. Throughput: 0: 1807.4, 1: 1799.2. Samples: 22922208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:34:53,442][51532] Avg episode reward: [(0, '45.280'), (1, '42.150')] -[2023-10-15 16:34:56,753][52833] Updated weights for policy 0, policy_version 44710 (0.0008) -[2023-10-15 16:34:57,139][52833] Updated weights for policy 0, policy_version 44720 (0.0007) -[2023-10-15 16:34:57,229][52866] Updated weights for policy 1, policy_version 44840 (0.0007) -[2023-10-15 16:34:57,510][52833] Updated weights for policy 0, policy_version 44730 (0.0007) -[2023-10-15 16:34:57,598][52866] Updated weights for policy 1, policy_version 44850 (0.0007) -[2023-10-15 16:34:57,958][52866] Updated weights for policy 1, policy_version 44860 (0.0007) -[2023-10-15 16:34:58,441][51532] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 91750400. Throughput: 0: 1793.7, 1: 1789.1. Samples: 22941828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:34:58,442][51532] Avg episode reward: [(0, '45.790'), (1, '40.930')] -[2023-10-15 16:35:01,352][52833] Updated weights for policy 0, policy_version 44740 (0.0008) -[2023-10-15 16:35:01,626][52866] Updated weights for policy 1, policy_version 44870 (0.0009) -[2023-10-15 16:35:01,721][52833] Updated weights for policy 0, policy_version 44750 (0.0007) -[2023-10-15 16:35:01,985][52866] Updated weights for policy 1, policy_version 44880 (0.0007) -[2023-10-15 16:35:02,084][52833] Updated weights for policy 0, policy_version 44760 (0.0008) -[2023-10-15 16:35:02,358][52866] Updated weights for policy 1, policy_version 44890 (0.0007) -[2023-10-15 16:35:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 91815936. Throughput: 0: 1800.8, 1: 1792.3. Samples: 22954350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:03,442][51532] Avg episode reward: [(0, '42.770'), (1, '40.220')] -[2023-10-15 16:35:05,826][52833] Updated weights for policy 0, policy_version 44770 (0.0009) -[2023-10-15 16:35:06,178][52866] Updated weights for policy 1, policy_version 44900 (0.0008) -[2023-10-15 16:35:06,194][52833] Updated weights for policy 0, policy_version 44780 (0.0007) -[2023-10-15 16:35:06,554][52866] Updated weights for policy 1, policy_version 44910 (0.0009) -[2023-10-15 16:35:06,564][52833] Updated weights for policy 0, policy_version 44790 (0.0009) -[2023-10-15 16:35:06,912][52866] Updated weights for policy 1, policy_version 44920 (0.0008) -[2023-10-15 16:35:06,923][52833] Updated weights for policy 0, policy_version 44800 (0.0007) -[2023-10-15 16:35:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 91881472. Throughput: 0: 1794.4, 1: 1793.4. Samples: 22974238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:08,442][51532] Avg episode reward: [(0, '43.450'), (1, '39.440')] -[2023-10-15 16:35:10,822][52866] Updated weights for policy 1, policy_version 44930 (0.0009) -[2023-10-15 16:35:10,845][52833] Updated weights for policy 0, policy_version 44810 (0.0008) -[2023-10-15 16:35:11,184][52866] Updated weights for policy 1, policy_version 44940 (0.0008) -[2023-10-15 16:35:11,216][52833] Updated weights for policy 0, policy_version 44820 (0.0009) -[2023-10-15 16:35:11,558][52866] Updated weights for policy 1, policy_version 44950 (0.0009) -[2023-10-15 16:35:11,588][52833] Updated weights for policy 0, policy_version 44830 (0.0008) -[2023-10-15 16:35:11,919][52866] Updated weights for policy 1, policy_version 44960 (0.0007) -[2023-10-15 16:35:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91947008. Throughput: 0: 1784.5, 1: 1781.7. Samples: 22995960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:13,442][51532] Avg episode reward: [(0, '42.920'), (1, '38.960')] -[2023-10-15 16:35:15,339][52833] Updated weights for policy 0, policy_version 44840 (0.0008) -[2023-10-15 16:35:15,576][52866] Updated weights for policy 1, policy_version 44970 (0.0007) -[2023-10-15 16:35:15,714][52833] Updated weights for policy 0, policy_version 44850 (0.0008) -[2023-10-15 16:35:15,947][52866] Updated weights for policy 1, policy_version 44980 (0.0009) -[2023-10-15 16:35:16,085][52833] Updated weights for policy 0, policy_version 44860 (0.0008) -[2023-10-15 16:35:16,317][52866] Updated weights for policy 1, policy_version 44990 (0.0010) -[2023-10-15 16:35:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 92012544. Throughput: 0: 1794.3, 1: 1793.8. Samples: 23006540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:18,442][51532] Avg episode reward: [(0, '45.560'), (1, '42.600')] -[2023-10-15 16:35:19,907][52833] Updated weights for policy 0, policy_version 44870 (0.0009) -[2023-10-15 16:35:20,012][52866] Updated weights for policy 1, policy_version 45000 (0.0008) -[2023-10-15 16:35:20,268][52833] Updated weights for policy 0, policy_version 44880 (0.0008) -[2023-10-15 16:35:20,380][52866] Updated weights for policy 1, policy_version 45010 (0.0008) -[2023-10-15 16:35:20,629][52833] Updated weights for policy 0, policy_version 44890 (0.0009) -[2023-10-15 16:35:20,754][52866] Updated weights for policy 1, policy_version 45020 (0.0008) -[2023-10-15 16:35:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 92078080. Throughput: 0: 1775.9, 1: 1784.5. Samples: 23027780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:23,442][51532] Avg episode reward: [(0, '48.310'), (1, '40.190')] -[2023-10-15 16:35:24,647][52833] Updated weights for policy 0, policy_version 44900 (0.0008) -[2023-10-15 16:35:24,711][52866] Updated weights for policy 1, policy_version 45030 (0.0007) -[2023-10-15 16:35:25,003][52833] Updated weights for policy 0, policy_version 44910 (0.0010) -[2023-10-15 16:35:25,074][52866] Updated weights for policy 1, policy_version 45040 (0.0010) -[2023-10-15 16:35:25,383][52833] Updated weights for policy 0, policy_version 44920 (0.0008) -[2023-10-15 16:35:25,440][52866] Updated weights for policy 1, policy_version 45050 (0.0007) -[2023-10-15 16:35:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 92143616. Throughput: 0: 1770.3, 1: 1780.4. Samples: 23049958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:28,442][51532] Avg episode reward: [(0, '50.260'), (1, '41.320')] -[2023-10-15 16:35:28,453][52410] Saving new best policy, reward=50.260! -[2023-10-15 16:35:29,059][52833] Updated weights for policy 0, policy_version 44930 (0.0008) -[2023-10-15 16:35:29,385][52866] Updated weights for policy 1, policy_version 45060 (0.0007) -[2023-10-15 16:35:29,431][52833] Updated weights for policy 0, policy_version 44940 (0.0007) -[2023-10-15 16:35:29,771][52866] Updated weights for policy 1, policy_version 45070 (0.0007) -[2023-10-15 16:35:29,793][52833] Updated weights for policy 0, policy_version 44950 (0.0007) -[2023-10-15 16:35:30,138][52866] Updated weights for policy 1, policy_version 45080 (0.0007) -[2023-10-15 16:35:30,165][52833] Updated weights for policy 0, policy_version 44960 (0.0007) -[2023-10-15 16:35:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 92209152. Throughput: 0: 1771.4, 1: 1781.6. Samples: 23059712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:33,442][51532] Avg episode reward: [(0, '50.400'), (1, '41.340')] -[2023-10-15 16:35:33,795][52866] Updated weights for policy 1, policy_version 45090 (0.0007) -[2023-10-15 16:35:33,887][52833] Updated weights for policy 0, policy_version 44970 (0.0008) -[2023-10-15 16:35:34,151][52866] Updated weights for policy 1, policy_version 45100 (0.0008) -[2023-10-15 16:35:34,256][52833] Updated weights for policy 0, policy_version 44980 (0.0007) -[2023-10-15 16:35:34,516][52866] Updated weights for policy 1, policy_version 45110 (0.0010) -[2023-10-15 16:35:34,630][52833] Updated weights for policy 0, policy_version 44990 (0.0008) -[2023-10-15 16:35:34,703][52410] Saving new best policy, reward=50.400! -[2023-10-15 16:35:34,877][52866] Updated weights for policy 1, policy_version 45120 (0.0007) -[2023-10-15 16:35:38,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 92274688. Throughput: 0: 1766.1, 1: 1786.1. Samples: 23082058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:38,441][51532] Avg episode reward: [(0, '49.830'), (1, '40.150')] -[2023-10-15 16:35:38,448][52833] Updated weights for policy 0, policy_version 45000 (0.0008) -[2023-10-15 16:35:38,588][52866] Updated weights for policy 1, policy_version 45130 (0.0009) -[2023-10-15 16:35:38,815][52833] Updated weights for policy 0, policy_version 45010 (0.0007) -[2023-10-15 16:35:38,949][52866] Updated weights for policy 1, policy_version 45140 (0.0009) -[2023-10-15 16:35:39,177][52833] Updated weights for policy 0, policy_version 45020 (0.0009) -[2023-10-15 16:35:39,308][52866] Updated weights for policy 1, policy_version 45150 (0.0007) -[2023-10-15 16:35:43,018][52833] Updated weights for policy 0, policy_version 45030 (0.0008) -[2023-10-15 16:35:43,022][52866] Updated weights for policy 1, policy_version 45160 (0.0007) -[2023-10-15 16:35:43,381][52866] Updated weights for policy 1, policy_version 45170 (0.0008) -[2023-10-15 16:35:43,396][52833] Updated weights for policy 0, policy_version 45040 (0.0008) -[2023-10-15 16:35:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 92340224. Throughput: 0: 1793.7, 1: 1812.8. Samples: 23104122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:43,442][51532] Avg episode reward: [(0, '49.860'), (1, '41.470')] -[2023-10-15 16:35:43,752][52866] Updated weights for policy 1, policy_version 45180 (0.0008) -[2023-10-15 16:35:43,755][52833] Updated weights for policy 0, policy_version 45050 (0.0009) -[2023-10-15 16:35:43,889][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000045184_46268416.pth... -[2023-10-15 16:35:43,922][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000043488_44531712.pth -[2023-10-15 16:35:43,973][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000045056_46137344.pth... -[2023-10-15 16:35:44,002][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000043360_44400640.pth -[2023-10-15 16:35:47,439][52833] Updated weights for policy 0, policy_version 45060 (0.0008) -[2023-10-15 16:35:47,587][52866] Updated weights for policy 1, policy_version 45190 (0.0009) -[2023-10-15 16:35:47,813][52833] Updated weights for policy 0, policy_version 45070 (0.0009) -[2023-10-15 16:35:47,957][52866] Updated weights for policy 1, policy_version 45200 (0.0009) -[2023-10-15 16:35:48,186][52833] Updated weights for policy 0, policy_version 45080 (0.0008) -[2023-10-15 16:35:48,324][52866] Updated weights for policy 1, policy_version 45210 (0.0009) -[2023-10-15 16:35:48,442][51532] Fps is (10 sec: 13105.9, 60 sec: 14199.2, 300 sec: 14217.9). Total num frames: 92405760. Throughput: 0: 1763.0, 1: 1787.9. Samples: 23114146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:48,443][51532] Avg episode reward: [(0, '48.940'), (1, '42.160')] -[2023-10-15 16:35:51,987][52866] Updated weights for policy 1, policy_version 45220 (0.0009) -[2023-10-15 16:35:52,072][52833] Updated weights for policy 0, policy_version 45090 (0.0009) -[2023-10-15 16:35:52,365][52866] Updated weights for policy 1, policy_version 45230 (0.0009) -[2023-10-15 16:35:52,437][52833] Updated weights for policy 0, policy_version 45100 (0.0008) -[2023-10-15 16:35:52,724][52866] Updated weights for policy 1, policy_version 45240 (0.0008) -[2023-10-15 16:35:52,808][52833] Updated weights for policy 0, policy_version 45110 (0.0008) -[2023-10-15 16:35:53,179][52833] Updated weights for policy 0, policy_version 45120 (0.0008) -[2023-10-15 16:35:53,441][51532] Fps is (10 sec: 19661.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 92536832. Throughput: 0: 1792.3, 1: 1809.5. Samples: 23136316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:35:53,442][51532] Avg episode reward: [(0, '52.030'), (1, '44.580')] -[2023-10-15 16:35:53,442][52410] Saving new best policy, reward=52.030! -[2023-10-15 16:35:56,489][52866] Updated weights for policy 1, policy_version 45250 (0.0009) -[2023-10-15 16:35:56,861][52866] Updated weights for policy 1, policy_version 45260 (0.0009) -[2023-10-15 16:35:56,885][52833] Updated weights for policy 0, policy_version 45130 (0.0007) -[2023-10-15 16:35:57,222][52866] Updated weights for policy 1, policy_version 45270 (0.0008) -[2023-10-15 16:35:57,250][52833] Updated weights for policy 0, policy_version 45140 (0.0008) -[2023-10-15 16:35:57,590][52866] Updated weights for policy 1, policy_version 45280 (0.0007) -[2023-10-15 16:35:57,624][52833] Updated weights for policy 0, policy_version 45150 (0.0010) -[2023-10-15 16:35:58,441][51532] Fps is (10 sec: 19662.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 92602368. Throughput: 0: 1769.6, 1: 1788.3. Samples: 23156068. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-15 16:35:58,442][51532] Avg episode reward: [(0, '51.920'), (1, '46.090')] -[2023-10-15 16:36:01,311][52866] Updated weights for policy 1, policy_version 45290 (0.0008) -[2023-10-15 16:36:01,444][52833] Updated weights for policy 0, policy_version 45160 (0.0008) -[2023-10-15 16:36:01,676][52866] Updated weights for policy 1, policy_version 45300 (0.0009) -[2023-10-15 16:36:01,821][52833] Updated weights for policy 0, policy_version 45170 (0.0008) -[2023-10-15 16:36:02,047][52866] Updated weights for policy 1, policy_version 45310 (0.0007) -[2023-10-15 16:36:02,182][52833] Updated weights for policy 0, policy_version 45180 (0.0007) -[2023-10-15 16:36:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 92667904. Throughput: 0: 1791.3, 1: 1812.7. Samples: 23168720. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-15 16:36:03,442][51532] Avg episode reward: [(0, '47.970'), (1, '43.390')] -[2023-10-15 16:36:05,726][52866] Updated weights for policy 1, policy_version 45320 (0.0008) -[2023-10-15 16:36:05,927][52833] Updated weights for policy 0, policy_version 45190 (0.0008) -[2023-10-15 16:36:06,097][52866] Updated weights for policy 1, policy_version 45330 (0.0007) -[2023-10-15 16:36:06,302][52833] Updated weights for policy 0, policy_version 45200 (0.0007) -[2023-10-15 16:36:06,461][52866] Updated weights for policy 1, policy_version 45340 (0.0008) -[2023-10-15 16:36:06,669][52833] Updated weights for policy 0, policy_version 45210 (0.0007) -[2023-10-15 16:36:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 92733440. Throughput: 0: 1775.9, 1: 1793.8. Samples: 23188416. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-15 16:36:08,442][51532] Avg episode reward: [(0, '45.920'), (1, '43.220')] -[2023-10-15 16:36:10,181][52866] Updated weights for policy 1, policy_version 45350 (0.0007) -[2023-10-15 16:36:10,437][52833] Updated weights for policy 0, policy_version 45220 (0.0010) -[2023-10-15 16:36:10,547][52866] Updated weights for policy 1, policy_version 45360 (0.0008) -[2023-10-15 16:36:10,801][52833] Updated weights for policy 0, policy_version 45230 (0.0010) -[2023-10-15 16:36:10,911][52866] Updated weights for policy 1, policy_version 45370 (0.0007) -[2023-10-15 16:36:11,165][52833] Updated weights for policy 0, policy_version 45240 (0.0008) -[2023-10-15 16:36:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 92798976. Throughput: 0: 1773.6, 1: 1798.1. Samples: 23210684. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-15 16:36:13,442][51532] Avg episode reward: [(0, '46.250'), (1, '45.440')] -[2023-10-15 16:36:14,820][52866] Updated weights for policy 1, policy_version 45380 (0.0009) -[2023-10-15 16:36:15,012][52833] Updated weights for policy 0, policy_version 45250 (0.0007) -[2023-10-15 16:36:15,200][52866] Updated weights for policy 1, policy_version 45390 (0.0008) -[2023-10-15 16:36:15,387][52833] Updated weights for policy 0, policy_version 45260 (0.0008) -[2023-10-15 16:36:15,559][52866] Updated weights for policy 1, policy_version 45400 (0.0008) -[2023-10-15 16:36:15,749][52833] Updated weights for policy 0, policy_version 45270 (0.0008) -[2023-10-15 16:36:16,123][52833] Updated weights for policy 0, policy_version 45280 (0.0008) -[2023-10-15 16:36:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 92864512. Throughput: 0: 1778.4, 1: 1793.5. Samples: 23220448. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-15 16:36:18,442][51532] Avg episode reward: [(0, '46.440'), (1, '44.490')] -[2023-10-15 16:36:19,356][52866] Updated weights for policy 1, policy_version 45410 (0.0007) -[2023-10-15 16:36:19,636][52833] Updated weights for policy 0, policy_version 45290 (0.0007) -[2023-10-15 16:36:19,714][52866] Updated weights for policy 1, policy_version 45420 (0.0009) -[2023-10-15 16:36:19,998][52833] Updated weights for policy 0, policy_version 45300 (0.0008) -[2023-10-15 16:36:20,084][52866] Updated weights for policy 1, policy_version 45430 (0.0009) -[2023-10-15 16:36:20,366][52833] Updated weights for policy 0, policy_version 45310 (0.0008) -[2023-10-15 16:36:20,450][52866] Updated weights for policy 1, policy_version 45440 (0.0007) -[2023-10-15 16:36:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 92930048. Throughput: 0: 1778.2, 1: 1788.0. Samples: 23242538. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-15 16:36:23,442][51532] Avg episode reward: [(0, '46.120'), (1, '46.330')] -[2023-10-15 16:36:24,081][52833] Updated weights for policy 0, policy_version 45320 (0.0008) -[2023-10-15 16:36:24,182][52866] Updated weights for policy 1, policy_version 45450 (0.0009) -[2023-10-15 16:36:24,446][52833] Updated weights for policy 0, policy_version 45330 (0.0008) -[2023-10-15 16:36:24,543][52866] Updated weights for policy 1, policy_version 45460 (0.0007) -[2023-10-15 16:36:24,804][52833] Updated weights for policy 0, policy_version 45340 (0.0009) -[2023-10-15 16:36:24,906][52866] Updated weights for policy 1, policy_version 45470 (0.0010) -[2023-10-15 16:36:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 92995584. Throughput: 0: 1785.2, 1: 1795.2. Samples: 23265242. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-15 16:36:28,442][51532] Avg episode reward: [(0, '46.360'), (1, '43.710')] -[2023-10-15 16:36:28,672][52866] Updated weights for policy 1, policy_version 45480 (0.0007) -[2023-10-15 16:36:28,688][52833] Updated weights for policy 0, policy_version 45350 (0.0009) -[2023-10-15 16:36:29,037][52866] Updated weights for policy 1, policy_version 45490 (0.0008) -[2023-10-15 16:36:29,051][52833] Updated weights for policy 0, policy_version 45360 (0.0009) -[2023-10-15 16:36:29,398][52866] Updated weights for policy 1, policy_version 45500 (0.0010) -[2023-10-15 16:36:29,418][52833] Updated weights for policy 0, policy_version 45370 (0.0007) -[2023-10-15 16:36:33,110][52866] Updated weights for policy 1, policy_version 45510 (0.0007) -[2023-10-15 16:36:33,227][52833] Updated weights for policy 0, policy_version 45380 (0.0008) -[2023-10-15 16:36:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 93061120. Throughput: 0: 1777.2, 1: 1792.1. Samples: 23274760. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-15 16:36:33,441][51532] Avg episode reward: [(0, '45.770'), (1, '42.960')] -[2023-10-15 16:36:33,470][52866] Updated weights for policy 1, policy_version 45520 (0.0007) -[2023-10-15 16:36:33,601][52833] Updated weights for policy 0, policy_version 45390 (0.0008) -[2023-10-15 16:36:33,842][52866] Updated weights for policy 1, policy_version 45530 (0.0009) -[2023-10-15 16:36:33,968][52833] Updated weights for policy 0, policy_version 45400 (0.0008) -[2023-10-15 16:36:37,716][52866] Updated weights for policy 1, policy_version 45540 (0.0008) -[2023-10-15 16:36:37,790][52833] Updated weights for policy 0, policy_version 45410 (0.0009) -[2023-10-15 16:36:38,074][52866] Updated weights for policy 1, policy_version 45550 (0.0007) -[2023-10-15 16:36:38,166][52833] Updated weights for policy 0, policy_version 45420 (0.0008) -[2023-10-15 16:36:38,433][52866] Updated weights for policy 1, policy_version 45560 (0.0007) -[2023-10-15 16:36:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 93126656. Throughput: 0: 1784.0, 1: 1790.5. Samples: 23297170. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-15 16:36:38,441][51532] Avg episode reward: [(0, '45.890'), (1, '43.240')] -[2023-10-15 16:36:38,538][52833] Updated weights for policy 0, policy_version 45430 (0.0007) -[2023-10-15 16:36:38,902][52833] Updated weights for policy 0, policy_version 45440 (0.0008) -[2023-10-15 16:36:42,187][52866] Updated weights for policy 1, policy_version 45570 (0.0008) -[2023-10-15 16:36:42,554][52866] Updated weights for policy 1, policy_version 45580 (0.0008) -[2023-10-15 16:36:42,680][52833] Updated weights for policy 0, policy_version 45450 (0.0009) -[2023-10-15 16:36:42,924][52866] Updated weights for policy 1, policy_version 45590 (0.0008) -[2023-10-15 16:36:43,046][52833] Updated weights for policy 0, policy_version 45460 (0.0009) -[2023-10-15 16:36:43,283][52866] Updated weights for policy 1, policy_version 45600 (0.0007) -[2023-10-15 16:36:43,410][52833] Updated weights for policy 0, policy_version 45470 (0.0009) -[2023-10-15 16:36:43,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 93224960. Throughput: 0: 1795.1, 1: 1800.2. Samples: 23317856. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-15 16:36:43,442][51532] Avg episode reward: [(0, '46.780'), (1, '40.740')] -[2023-10-15 16:36:46,986][52866] Updated weights for policy 1, policy_version 45610 (0.0009) -[2023-10-15 16:36:47,188][52833] Updated weights for policy 0, policy_version 45480 (0.0009) -[2023-10-15 16:36:47,356][52866] Updated weights for policy 1, policy_version 45620 (0.0009) -[2023-10-15 16:36:47,551][52833] Updated weights for policy 0, policy_version 45490 (0.0007) -[2023-10-15 16:36:47,726][52866] Updated weights for policy 1, policy_version 45630 (0.0009) -[2023-10-15 16:36:47,929][52833] Updated weights for policy 0, policy_version 45500 (0.0008) -[2023-10-15 16:36:48,441][51532] Fps is (10 sec: 19660.7, 60 sec: 15292.0, 300 sec: 14440.1). Total num frames: 93323264. Throughput: 0: 1780.8, 1: 1791.9. Samples: 23329490. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-15 16:36:48,442][51532] Avg episode reward: [(0, '48.250'), (1, '43.750')] -[2023-10-15 16:36:51,553][52866] Updated weights for policy 1, policy_version 45640 (0.0008) -[2023-10-15 16:36:51,614][52833] Updated weights for policy 0, policy_version 45510 (0.0008) -[2023-10-15 16:36:51,915][52866] Updated weights for policy 1, policy_version 45650 (0.0008) -[2023-10-15 16:36:51,986][52833] Updated weights for policy 0, policy_version 45520 (0.0007) -[2023-10-15 16:36:52,267][52866] Updated weights for policy 1, policy_version 45660 (0.0007) -[2023-10-15 16:36:52,359][52833] Updated weights for policy 0, policy_version 45530 (0.0007) -[2023-10-15 16:36:53,441][51532] Fps is (10 sec: 16384.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 93388800. Throughput: 0: 1800.0, 1: 1795.3. Samples: 23350204. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-15 16:36:53,441][51532] Avg episode reward: [(0, '46.130'), (1, '43.950')] -[2023-10-15 16:36:56,042][52866] Updated weights for policy 1, policy_version 45670 (0.0009) -[2023-10-15 16:36:56,218][52833] Updated weights for policy 0, policy_version 45540 (0.0009) -[2023-10-15 16:36:56,404][52866] Updated weights for policy 1, policy_version 45680 (0.0008) -[2023-10-15 16:36:56,584][52833] Updated weights for policy 0, policy_version 45550 (0.0010) -[2023-10-15 16:36:56,762][52866] Updated weights for policy 1, policy_version 45690 (0.0008) -[2023-10-15 16:36:56,959][52833] Updated weights for policy 0, policy_version 45560 (0.0008) -[2023-10-15 16:36:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 93454336. Throughput: 0: 1778.4, 1: 1780.6. Samples: 23370838. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-15 16:36:58,441][51532] Avg episode reward: [(0, '46.840'), (1, '41.990')] -[2023-10-15 16:37:00,455][52866] Updated weights for policy 1, policy_version 45700 (0.0008) -[2023-10-15 16:37:00,822][52866] Updated weights for policy 1, policy_version 45710 (0.0008) -[2023-10-15 16:37:00,895][52833] Updated weights for policy 0, policy_version 45570 (0.0009) -[2023-10-15 16:37:01,186][52866] Updated weights for policy 1, policy_version 45720 (0.0009) -[2023-10-15 16:37:01,270][52833] Updated weights for policy 0, policy_version 45580 (0.0007) -[2023-10-15 16:37:01,633][52833] Updated weights for policy 0, policy_version 45590 (0.0009) -[2023-10-15 16:37:02,001][52833] Updated weights for policy 0, policy_version 45600 (0.0008) -[2023-10-15 16:37:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93519872. Throughput: 0: 1802.5, 1: 1807.4. Samples: 23382892. Policy #0 lag: (min: 22.0, avg: 22.2, max: 33.0) -[2023-10-15 16:37:03,441][51532] Avg episode reward: [(0, '45.300'), (1, '40.700')] -[2023-10-15 16:37:04,855][52866] Updated weights for policy 1, policy_version 45730 (0.0008) -[2023-10-15 16:37:05,228][52866] Updated weights for policy 1, policy_version 45740 (0.0007) -[2023-10-15 16:37:05,583][52866] Updated weights for policy 1, policy_version 45750 (0.0007) -[2023-10-15 16:37:05,845][52833] Updated weights for policy 0, policy_version 45610 (0.0008) -[2023-10-15 16:37:05,952][52866] Updated weights for policy 1, policy_version 45760 (0.0008) -[2023-10-15 16:37:06,212][52833] Updated weights for policy 0, policy_version 45620 (0.0008) -[2023-10-15 16:37:06,580][52833] Updated weights for policy 0, policy_version 45630 (0.0008) -[2023-10-15 16:37:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 93585408. Throughput: 0: 1774.4, 1: 1798.0. Samples: 23403296. Policy #0 lag: (min: 22.0, avg: 22.2, max: 33.0) -[2023-10-15 16:37:08,442][51532] Avg episode reward: [(0, '45.480'), (1, '42.440')] -[2023-10-15 16:37:09,730][52866] Updated weights for policy 1, policy_version 45770 (0.0010) -[2023-10-15 16:37:10,103][52866] Updated weights for policy 1, policy_version 45780 (0.0008) -[2023-10-15 16:37:10,219][52833] Updated weights for policy 0, policy_version 45640 (0.0009) -[2023-10-15 16:37:10,462][52866] Updated weights for policy 1, policy_version 45790 (0.0008) -[2023-10-15 16:37:10,582][52833] Updated weights for policy 0, policy_version 45650 (0.0008) -[2023-10-15 16:37:10,962][52833] Updated weights for policy 0, policy_version 45660 (0.0007) -[2023-10-15 16:37:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 93650944. Throughput: 0: 1776.3, 1: 1797.2. Samples: 23426046. Policy #0 lag: (min: 22.0, avg: 22.2, max: 33.0) -[2023-10-15 16:37:13,442][51532] Avg episode reward: [(0, '44.560'), (1, '43.690')] -[2023-10-15 16:37:14,089][52866] Updated weights for policy 1, policy_version 45800 (0.0008) -[2023-10-15 16:37:14,453][52866] Updated weights for policy 1, policy_version 45810 (0.0008) -[2023-10-15 16:37:14,718][52833] Updated weights for policy 0, policy_version 45670 (0.0008) -[2023-10-15 16:37:14,810][52866] Updated weights for policy 1, policy_version 45820 (0.0009) -[2023-10-15 16:37:15,093][52833] Updated weights for policy 0, policy_version 45680 (0.0010) -[2023-10-15 16:37:15,461][52833] Updated weights for policy 0, policy_version 45690 (0.0010) -[2023-10-15 16:37:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 93716480. Throughput: 0: 1784.8, 1: 1798.9. Samples: 23436024. Policy #0 lag: (min: 22.0, avg: 22.2, max: 33.0) -[2023-10-15 16:37:18,441][51532] Avg episode reward: [(0, '44.400'), (1, '45.110')] -[2023-10-15 16:37:18,629][52866] Updated weights for policy 1, policy_version 45830 (0.0008) -[2023-10-15 16:37:18,992][52866] Updated weights for policy 1, policy_version 45840 (0.0008) -[2023-10-15 16:37:19,088][52833] Updated weights for policy 0, policy_version 45700 (0.0010) -[2023-10-15 16:37:19,357][52866] Updated weights for policy 1, policy_version 45850 (0.0007) -[2023-10-15 16:37:19,456][52833] Updated weights for policy 0, policy_version 45710 (0.0008) -[2023-10-15 16:37:19,813][52833] Updated weights for policy 0, policy_version 45720 (0.0008) -[2023-10-15 16:37:22,985][52866] Updated weights for policy 1, policy_version 45860 (0.0008) -[2023-10-15 16:37:23,349][52866] Updated weights for policy 1, policy_version 45870 (0.0008) -[2023-10-15 16:37:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 93782016. Throughput: 0: 1780.1, 1: 1807.4. Samples: 23458610. Policy #0 lag: (min: 22.0, avg: 22.2, max: 33.0) -[2023-10-15 16:37:23,442][51532] Avg episode reward: [(0, '42.560'), (1, '43.760')] -[2023-10-15 16:37:23,688][52833] Updated weights for policy 0, policy_version 45730 (0.0010) -[2023-10-15 16:37:23,717][52866] Updated weights for policy 1, policy_version 45880 (0.0009) -[2023-10-15 16:37:24,052][52833] Updated weights for policy 0, policy_version 45740 (0.0008) -[2023-10-15 16:37:24,425][52833] Updated weights for policy 0, policy_version 45750 (0.0009) -[2023-10-15 16:37:24,789][52833] Updated weights for policy 0, policy_version 45760 (0.0010) -[2023-10-15 16:37:27,560][52866] Updated weights for policy 1, policy_version 45890 (0.0009) -[2023-10-15 16:37:27,923][52866] Updated weights for policy 1, policy_version 45900 (0.0008) -[2023-10-15 16:37:28,293][52866] Updated weights for policy 1, policy_version 45910 (0.0007) -[2023-10-15 16:37:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 93847552. Throughput: 0: 1797.9, 1: 1813.7. Samples: 23480378. Policy #0 lag: (min: 22.0, avg: 22.2, max: 33.0) -[2023-10-15 16:37:28,442][51532] Avg episode reward: [(0, '43.700'), (1, '45.060')] -[2023-10-15 16:37:28,608][52833] Updated weights for policy 0, policy_version 45770 (0.0007) -[2023-10-15 16:37:28,656][52866] Updated weights for policy 1, policy_version 45920 (0.0009) -[2023-10-15 16:37:28,972][52833] Updated weights for policy 0, policy_version 45780 (0.0007) -[2023-10-15 16:37:29,347][52833] Updated weights for policy 0, policy_version 45790 (0.0008) -[2023-10-15 16:37:32,344][52866] Updated weights for policy 1, policy_version 45930 (0.0010) -[2023-10-15 16:37:32,707][52866] Updated weights for policy 1, policy_version 45940 (0.0011) -[2023-10-15 16:37:33,078][52866] Updated weights for policy 1, policy_version 45950 (0.0008) -[2023-10-15 16:37:33,091][52833] Updated weights for policy 0, policy_version 45800 (0.0008) -[2023-10-15 16:37:33,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 93945856. Throughput: 0: 1781.0, 1: 1801.1. Samples: 23490686. Policy #0 lag: (min: 22.0, avg: 22.2, max: 33.0) -[2023-10-15 16:37:33,441][51532] Avg episode reward: [(0, '43.440'), (1, '45.980')] -[2023-10-15 16:37:33,451][52833] Updated weights for policy 0, policy_version 45810 (0.0010) -[2023-10-15 16:37:33,815][52833] Updated weights for policy 0, policy_version 45820 (0.0009) -[2023-10-15 16:37:36,842][52866] Updated weights for policy 1, policy_version 45960 (0.0007) -[2023-10-15 16:37:37,207][52866] Updated weights for policy 1, policy_version 45970 (0.0008) -[2023-10-15 16:37:37,563][52866] Updated weights for policy 1, policy_version 45980 (0.0008) -[2023-10-15 16:37:37,575][52833] Updated weights for policy 0, policy_version 45830 (0.0009) -[2023-10-15 16:37:37,937][52833] Updated weights for policy 0, policy_version 45840 (0.0010) -[2023-10-15 16:37:38,297][52833] Updated weights for policy 0, policy_version 45850 (0.0011) -[2023-10-15 16:37:38,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 94011392. Throughput: 0: 1796.2, 1: 1815.1. Samples: 23512716. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 16:37:38,442][51532] Avg episode reward: [(0, '43.140'), (1, '45.290')] -[2023-10-15 16:37:41,320][52866] Updated weights for policy 1, policy_version 45990 (0.0009) -[2023-10-15 16:37:41,691][52866] Updated weights for policy 1, policy_version 46000 (0.0009) -[2023-10-15 16:37:41,983][52833] Updated weights for policy 0, policy_version 45860 (0.0007) -[2023-10-15 16:37:42,051][52866] Updated weights for policy 1, policy_version 46010 (0.0010) -[2023-10-15 16:37:42,362][52833] Updated weights for policy 0, policy_version 45870 (0.0009) -[2023-10-15 16:37:42,719][52833] Updated weights for policy 0, policy_version 45880 (0.0007) -[2023-10-15 16:37:43,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 94109696. Throughput: 0: 1791.7, 1: 1815.1. Samples: 23533146. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 16:37:43,442][51532] Avg episode reward: [(0, '45.250'), (1, '46.860')] -[2023-10-15 16:37:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000045888_46989312.pth... -[2023-10-15 16:37:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000046016_47120384.pth... -[2023-10-15 16:37:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000044224_45285376.pth -[2023-10-15 16:37:43,492][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000044320_45383680.pth -[2023-10-15 16:37:45,803][52866] Updated weights for policy 1, policy_version 46020 (0.0009) -[2023-10-15 16:37:46,198][52866] Updated weights for policy 1, policy_version 46030 (0.0007) -[2023-10-15 16:37:46,566][52866] Updated weights for policy 1, policy_version 46040 (0.0008) -[2023-10-15 16:37:46,603][52833] Updated weights for policy 0, policy_version 45890 (0.0008) -[2023-10-15 16:37:46,965][52833] Updated weights for policy 0, policy_version 45900 (0.0009) -[2023-10-15 16:37:47,339][52833] Updated weights for policy 0, policy_version 45910 (0.0009) -[2023-10-15 16:37:47,712][52833] Updated weights for policy 0, policy_version 45920 (0.0011) -[2023-10-15 16:37:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94175232. Throughput: 0: 1787.0, 1: 1814.5. Samples: 23544960. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 16:37:48,442][51532] Avg episode reward: [(0, '45.760'), (1, '50.520')] -[2023-10-15 16:37:50,215][52866] Updated weights for policy 1, policy_version 46050 (0.0007) -[2023-10-15 16:37:50,578][52866] Updated weights for policy 1, policy_version 46060 (0.0008) -[2023-10-15 16:37:50,942][52866] Updated weights for policy 1, policy_version 46070 (0.0009) -[2023-10-15 16:37:51,303][52866] Updated weights for policy 1, policy_version 46080 (0.0008) -[2023-10-15 16:37:51,559][52833] Updated weights for policy 0, policy_version 45930 (0.0010) -[2023-10-15 16:37:51,933][52833] Updated weights for policy 0, policy_version 45940 (0.0007) -[2023-10-15 16:37:52,294][52833] Updated weights for policy 0, policy_version 45950 (0.0007) -[2023-10-15 16:37:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 94240768. Throughput: 0: 1800.7, 1: 1806.6. Samples: 23565626. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 16:37:53,442][51532] Avg episode reward: [(0, '46.290'), (1, '49.910')] -[2023-10-15 16:37:55,061][52866] Updated weights for policy 1, policy_version 46090 (0.0010) -[2023-10-15 16:37:55,424][52866] Updated weights for policy 1, policy_version 46100 (0.0008) -[2023-10-15 16:37:55,794][52866] Updated weights for policy 1, policy_version 46110 (0.0008) -[2023-10-15 16:37:55,918][52833] Updated weights for policy 0, policy_version 45960 (0.0008) -[2023-10-15 16:37:56,287][52833] Updated weights for policy 0, policy_version 45970 (0.0009) -[2023-10-15 16:37:56,659][52833] Updated weights for policy 0, policy_version 45980 (0.0008) -[2023-10-15 16:37:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 94306304. Throughput: 0: 1782.3, 1: 1802.2. Samples: 23587350. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 16:37:58,442][51532] Avg episode reward: [(0, '50.300'), (1, '50.820')] -[2023-10-15 16:37:59,492][52866] Updated weights for policy 1, policy_version 46120 (0.0010) -[2023-10-15 16:37:59,865][52866] Updated weights for policy 1, policy_version 46130 (0.0011) -[2023-10-15 16:38:00,223][52866] Updated weights for policy 1, policy_version 46140 (0.0008) -[2023-10-15 16:38:00,550][52833] Updated weights for policy 0, policy_version 45990 (0.0009) -[2023-10-15 16:38:00,935][52833] Updated weights for policy 0, policy_version 46000 (0.0008) -[2023-10-15 16:38:01,298][52833] Updated weights for policy 0, policy_version 46010 (0.0008) -[2023-10-15 16:38:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 94371840. Throughput: 0: 1798.1, 1: 1795.6. Samples: 23597742. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 16:38:03,441][51532] Avg episode reward: [(0, '52.440'), (1, '51.920')] -[2023-10-15 16:38:03,442][52410] Saving new best policy, reward=52.440! -[2023-10-15 16:38:04,146][52866] Updated weights for policy 1, policy_version 46150 (0.0009) -[2023-10-15 16:38:04,518][52866] Updated weights for policy 1, policy_version 46160 (0.0008) -[2023-10-15 16:38:04,883][52866] Updated weights for policy 1, policy_version 46170 (0.0010) -[2023-10-15 16:38:05,054][52833] Updated weights for policy 0, policy_version 46020 (0.0008) -[2023-10-15 16:38:05,421][52833] Updated weights for policy 0, policy_version 46030 (0.0008) -[2023-10-15 16:38:05,791][52833] Updated weights for policy 0, policy_version 46040 (0.0008) -[2023-10-15 16:38:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 94437376. Throughput: 0: 1775.3, 1: 1789.2. Samples: 23619014. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 16:38:08,442][51532] Avg episode reward: [(0, '53.170'), (1, '51.240')] -[2023-10-15 16:38:08,444][52410] Saving new best policy, reward=53.170! -[2023-10-15 16:38:08,691][52866] Updated weights for policy 1, policy_version 46180 (0.0008) -[2023-10-15 16:38:09,053][52866] Updated weights for policy 1, policy_version 46190 (0.0010) -[2023-10-15 16:38:09,426][52866] Updated weights for policy 1, policy_version 46200 (0.0007) -[2023-10-15 16:38:09,482][52833] Updated weights for policy 0, policy_version 46050 (0.0010) -[2023-10-15 16:38:09,842][52833] Updated weights for policy 0, policy_version 46060 (0.0009) -[2023-10-15 16:38:10,211][52833] Updated weights for policy 0, policy_version 46070 (0.0008) -[2023-10-15 16:38:10,579][52833] Updated weights for policy 0, policy_version 46080 (0.0009) -[2023-10-15 16:38:13,268][52866] Updated weights for policy 1, policy_version 46210 (0.0009) -[2023-10-15 16:38:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 94502912. Throughput: 0: 1781.1, 1: 1798.7. Samples: 23641470. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 16:38:13,442][51532] Avg episode reward: [(0, '51.460'), (1, '52.350')] -[2023-10-15 16:38:13,637][52866] Updated weights for policy 1, policy_version 46220 (0.0009) -[2023-10-15 16:38:13,996][52866] Updated weights for policy 1, policy_version 46230 (0.0007) -[2023-10-15 16:38:14,339][52833] Updated weights for policy 0, policy_version 46090 (0.0009) -[2023-10-15 16:38:14,378][52866] Updated weights for policy 1, policy_version 46240 (0.0009) -[2023-10-15 16:38:14,712][52833] Updated weights for policy 0, policy_version 46100 (0.0009) -[2023-10-15 16:38:15,077][52833] Updated weights for policy 0, policy_version 46110 (0.0009) -[2023-10-15 16:38:18,069][52866] Updated weights for policy 1, policy_version 46250 (0.0007) -[2023-10-15 16:38:18,432][52866] Updated weights for policy 1, policy_version 46260 (0.0009) -[2023-10-15 16:38:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 94568448. Throughput: 0: 1783.4, 1: 1784.0. Samples: 23651218. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 16:38:18,442][51532] Avg episode reward: [(0, '53.600'), (1, '51.070')] -[2023-10-15 16:38:18,442][52410] Saving new best policy, reward=53.600! -[2023-10-15 16:38:18,805][52866] Updated weights for policy 1, policy_version 46270 (0.0008) -[2023-10-15 16:38:18,880][52833] Updated weights for policy 0, policy_version 46120 (0.0008) -[2023-10-15 16:38:19,255][52833] Updated weights for policy 0, policy_version 46130 (0.0009) -[2023-10-15 16:38:19,627][52833] Updated weights for policy 0, policy_version 46140 (0.0009) -[2023-10-15 16:38:22,469][52866] Updated weights for policy 1, policy_version 46280 (0.0009) -[2023-10-15 16:38:22,835][52866] Updated weights for policy 1, policy_version 46290 (0.0010) -[2023-10-15 16:38:23,202][52866] Updated weights for policy 1, policy_version 46300 (0.0010) -[2023-10-15 16:38:23,413][52833] Updated weights for policy 0, policy_version 46150 (0.0008) -[2023-10-15 16:38:23,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 94666752. Throughput: 0: 1780.8, 1: 1796.5. Samples: 23673694. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 16:38:23,441][51532] Avg episode reward: [(0, '53.020'), (1, '49.500')] -[2023-10-15 16:38:23,784][52833] Updated weights for policy 0, policy_version 46160 (0.0009) -[2023-10-15 16:38:24,153][52833] Updated weights for policy 0, policy_version 46170 (0.0010) -[2023-10-15 16:38:26,890][52866] Updated weights for policy 1, policy_version 46310 (0.0009) -[2023-10-15 16:38:27,259][52866] Updated weights for policy 1, policy_version 46320 (0.0008) -[2023-10-15 16:38:27,615][52866] Updated weights for policy 1, policy_version 46330 (0.0008) -[2023-10-15 16:38:27,837][52833] Updated weights for policy 0, policy_version 46180 (0.0008) -[2023-10-15 16:38:28,203][52833] Updated weights for policy 0, policy_version 46190 (0.0008) -[2023-10-15 16:38:28,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 94732288. Throughput: 0: 1808.4, 1: 1779.5. Samples: 23694602. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 16:38:28,441][51532] Avg episode reward: [(0, '52.650'), (1, '47.090')] -[2023-10-15 16:38:28,577][52833] Updated weights for policy 0, policy_version 46200 (0.0010) -[2023-10-15 16:38:31,361][52866] Updated weights for policy 1, policy_version 46340 (0.0008) -[2023-10-15 16:38:31,760][52866] Updated weights for policy 1, policy_version 46350 (0.0008) -[2023-10-15 16:38:32,129][52866] Updated weights for policy 1, policy_version 46360 (0.0007) -[2023-10-15 16:38:32,378][52833] Updated weights for policy 0, policy_version 46210 (0.0009) -[2023-10-15 16:38:32,753][52833] Updated weights for policy 0, policy_version 46220 (0.0010) -[2023-10-15 16:38:33,137][52833] Updated weights for policy 0, policy_version 46230 (0.0007) -[2023-10-15 16:38:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 94797824. Throughput: 0: 1782.6, 1: 1793.7. Samples: 23705894. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 16:38:33,441][51532] Avg episode reward: [(0, '55.410'), (1, '44.800')] -[2023-10-15 16:38:33,497][52410] Saving new best policy, reward=55.410! -[2023-10-15 16:38:33,497][52833] Updated weights for policy 0, policy_version 46240 (0.0008) -[2023-10-15 16:38:35,762][52866] Updated weights for policy 1, policy_version 46370 (0.0007) -[2023-10-15 16:38:36,125][52866] Updated weights for policy 1, policy_version 46380 (0.0008) -[2023-10-15 16:38:36,503][52866] Updated weights for policy 1, policy_version 46390 (0.0008) -[2023-10-15 16:38:36,861][52866] Updated weights for policy 1, policy_version 46400 (0.0007) -[2023-10-15 16:38:37,413][52833] Updated weights for policy 0, policy_version 46250 (0.0007) -[2023-10-15 16:38:37,780][52833] Updated weights for policy 0, policy_version 46260 (0.0008) -[2023-10-15 16:38:38,153][52833] Updated weights for policy 0, policy_version 46270 (0.0008) -[2023-10-15 16:38:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 94896128. Throughput: 0: 1805.6, 1: 1783.3. Samples: 23727128. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:38:38,441][51532] Avg episode reward: [(0, '52.200'), (1, '46.970')] -[2023-10-15 16:38:40,695][52866] Updated weights for policy 1, policy_version 46410 (0.0008) -[2023-10-15 16:38:41,062][52866] Updated weights for policy 1, policy_version 46420 (0.0008) -[2023-10-15 16:38:41,425][52866] Updated weights for policy 1, policy_version 46430 (0.0010) -[2023-10-15 16:38:41,693][52833] Updated weights for policy 0, policy_version 46280 (0.0007) -[2023-10-15 16:38:42,071][52833] Updated weights for policy 0, policy_version 46290 (0.0007) -[2023-10-15 16:38:42,438][52833] Updated weights for policy 0, policy_version 46300 (0.0007) -[2023-10-15 16:38:43,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94961664. Throughput: 0: 1796.5, 1: 1786.0. Samples: 23748560. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:38:43,442][51532] Avg episode reward: [(0, '50.080'), (1, '44.330')] -[2023-10-15 16:38:45,121][52866] Updated weights for policy 1, policy_version 46440 (0.0007) -[2023-10-15 16:38:45,481][52866] Updated weights for policy 1, policy_version 46450 (0.0007) -[2023-10-15 16:38:45,855][52866] Updated weights for policy 1, policy_version 46460 (0.0009) -[2023-10-15 16:38:46,229][52833] Updated weights for policy 0, policy_version 46310 (0.0007) -[2023-10-15 16:38:46,615][52833] Updated weights for policy 0, policy_version 46320 (0.0008) -[2023-10-15 16:38:46,986][52833] Updated weights for policy 0, policy_version 46330 (0.0008) -[2023-10-15 16:38:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95027200. Throughput: 0: 1812.6, 1: 1792.0. Samples: 23759952. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:38:48,441][51532] Avg episode reward: [(0, '49.750'), (1, '44.670')] -[2023-10-15 16:38:49,618][52866] Updated weights for policy 1, policy_version 46470 (0.0008) -[2023-10-15 16:38:49,978][52866] Updated weights for policy 1, policy_version 46480 (0.0007) -[2023-10-15 16:38:50,345][52866] Updated weights for policy 1, policy_version 46490 (0.0011) -[2023-10-15 16:38:50,654][52833] Updated weights for policy 0, policy_version 46340 (0.0010) -[2023-10-15 16:38:51,027][52833] Updated weights for policy 0, policy_version 46350 (0.0010) -[2023-10-15 16:38:51,399][52833] Updated weights for policy 0, policy_version 46360 (0.0008) -[2023-10-15 16:38:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 95092736. Throughput: 0: 1797.2, 1: 1792.6. Samples: 23780554. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:38:53,442][51532] Avg episode reward: [(0, '49.630'), (1, '44.260')] -[2023-10-15 16:38:54,062][52866] Updated weights for policy 1, policy_version 46500 (0.0008) -[2023-10-15 16:38:54,427][52866] Updated weights for policy 1, policy_version 46510 (0.0008) -[2023-10-15 16:38:54,801][52866] Updated weights for policy 1, policy_version 46520 (0.0007) -[2023-10-15 16:38:55,221][52833] Updated weights for policy 0, policy_version 46370 (0.0010) -[2023-10-15 16:38:55,591][52833] Updated weights for policy 0, policy_version 46380 (0.0010) -[2023-10-15 16:38:55,966][52833] Updated weights for policy 0, policy_version 46390 (0.0010) -[2023-10-15 16:38:56,325][52833] Updated weights for policy 0, policy_version 46400 (0.0008) -[2023-10-15 16:38:58,434][52866] Updated weights for policy 1, policy_version 46530 (0.0008) -[2023-10-15 16:38:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 95158272. Throughput: 0: 1785.3, 1: 1806.6. Samples: 23803104. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:38:58,441][51532] Avg episode reward: [(0, '49.280'), (1, '44.110')] -[2023-10-15 16:38:58,793][52866] Updated weights for policy 1, policy_version 46540 (0.0010) -[2023-10-15 16:38:59,159][52866] Updated weights for policy 1, policy_version 46550 (0.0009) -[2023-10-15 16:38:59,511][52866] Updated weights for policy 1, policy_version 46560 (0.0008) -[2023-10-15 16:39:00,125][52833] Updated weights for policy 0, policy_version 46410 (0.0010) -[2023-10-15 16:39:00,498][52833] Updated weights for policy 0, policy_version 46420 (0.0011) -[2023-10-15 16:39:00,857][52833] Updated weights for policy 0, policy_version 46430 (0.0010) -[2023-10-15 16:39:03,219][52866] Updated weights for policy 1, policy_version 46570 (0.0009) -[2023-10-15 16:39:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 95223808. Throughput: 0: 1789.0, 1: 1811.3. Samples: 23813234. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:39:03,442][51532] Avg episode reward: [(0, '52.370'), (1, '44.570')] -[2023-10-15 16:39:03,592][52866] Updated weights for policy 1, policy_version 46580 (0.0007) -[2023-10-15 16:39:03,955][52866] Updated weights for policy 1, policy_version 46590 (0.0008) -[2023-10-15 16:39:04,567][52833] Updated weights for policy 0, policy_version 46440 (0.0009) -[2023-10-15 16:39:04,939][52833] Updated weights for policy 0, policy_version 46450 (0.0008) -[2023-10-15 16:39:05,309][52833] Updated weights for policy 0, policy_version 46460 (0.0007) -[2023-10-15 16:39:07,630][52866] Updated weights for policy 1, policy_version 46600 (0.0008) -[2023-10-15 16:39:07,990][52866] Updated weights for policy 1, policy_version 46610 (0.0008) -[2023-10-15 16:39:08,367][52866] Updated weights for policy 1, policy_version 46620 (0.0008) -[2023-10-15 16:39:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 95289344. Throughput: 0: 1782.8, 1: 1813.8. Samples: 23835540. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 16:39:08,442][51532] Avg episode reward: [(0, '52.680'), (1, '45.280')] -[2023-10-15 16:39:09,032][52833] Updated weights for policy 0, policy_version 46470 (0.0008) -[2023-10-15 16:39:09,402][52833] Updated weights for policy 0, policy_version 46480 (0.0007) -[2023-10-15 16:39:09,765][52833] Updated weights for policy 0, policy_version 46490 (0.0008) -[2023-10-15 16:39:12,175][52866] Updated weights for policy 1, policy_version 46630 (0.0008) -[2023-10-15 16:39:12,547][52866] Updated weights for policy 1, policy_version 46640 (0.0008) -[2023-10-15 16:39:12,921][52866] Updated weights for policy 1, policy_version 46650 (0.0008) -[2023-10-15 16:39:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 95387648. Throughput: 0: 1789.2, 1: 1818.3. Samples: 23856942. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 16:39:13,442][51532] Avg episode reward: [(0, '53.260'), (1, '44.940')] -[2023-10-15 16:39:13,504][52833] Updated weights for policy 0, policy_version 46500 (0.0007) -[2023-10-15 16:39:13,872][52833] Updated weights for policy 0, policy_version 46510 (0.0008) -[2023-10-15 16:39:14,248][52833] Updated weights for policy 0, policy_version 46520 (0.0008) -[2023-10-15 16:39:16,765][52866] Updated weights for policy 1, policy_version 46660 (0.0007) -[2023-10-15 16:39:17,151][52866] Updated weights for policy 1, policy_version 46670 (0.0007) -[2023-10-15 16:39:17,509][52866] Updated weights for policy 1, policy_version 46680 (0.0007) -[2023-10-15 16:39:18,242][52833] Updated weights for policy 0, policy_version 46530 (0.0009) -[2023-10-15 16:39:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 95453184. Throughput: 0: 1786.5, 1: 1813.9. Samples: 23867916. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 16:39:18,442][51532] Avg episode reward: [(0, '51.240'), (1, '45.100')] -[2023-10-15 16:39:18,615][52833] Updated weights for policy 0, policy_version 46540 (0.0008) -[2023-10-15 16:39:18,976][52833] Updated weights for policy 0, policy_version 46550 (0.0008) -[2023-10-15 16:39:19,354][52833] Updated weights for policy 0, policy_version 46560 (0.0007) -[2023-10-15 16:39:21,244][52866] Updated weights for policy 1, policy_version 46690 (0.0010) -[2023-10-15 16:39:21,619][52866] Updated weights for policy 1, policy_version 46700 (0.0011) -[2023-10-15 16:39:21,987][52866] Updated weights for policy 1, policy_version 46710 (0.0010) -[2023-10-15 16:39:22,354][52866] Updated weights for policy 1, policy_version 46720 (0.0010) -[2023-10-15 16:39:22,991][52833] Updated weights for policy 0, policy_version 46570 (0.0007) -[2023-10-15 16:39:23,352][52833] Updated weights for policy 0, policy_version 46580 (0.0008) -[2023-10-15 16:39:23,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 95518720. Throughput: 0: 1780.9, 1: 1817.0. Samples: 23889032. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 16:39:23,442][51532] Avg episode reward: [(0, '53.210'), (1, '43.120')] -[2023-10-15 16:39:23,724][52833] Updated weights for policy 0, policy_version 46590 (0.0009) -[2023-10-15 16:39:26,035][52866] Updated weights for policy 1, policy_version 46730 (0.0009) -[2023-10-15 16:39:26,403][52866] Updated weights for policy 1, policy_version 46740 (0.0010) -[2023-10-15 16:39:26,781][52866] Updated weights for policy 1, policy_version 46750 (0.0010) -[2023-10-15 16:39:27,495][52833] Updated weights for policy 0, policy_version 46600 (0.0008) -[2023-10-15 16:39:27,864][52833] Updated weights for policy 0, policy_version 46610 (0.0011) -[2023-10-15 16:39:28,232][52833] Updated weights for policy 0, policy_version 46620 (0.0009) -[2023-10-15 16:39:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 95617024. Throughput: 0: 1787.5, 1: 1807.1. Samples: 23910316. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 16:39:28,442][51532] Avg episode reward: [(0, '56.580'), (1, '45.230')] -[2023-10-15 16:39:28,454][52410] Saving new best policy, reward=56.580! -[2023-10-15 16:39:30,378][52866] Updated weights for policy 1, policy_version 46760 (0.0009) -[2023-10-15 16:39:30,755][52866] Updated weights for policy 1, policy_version 46770 (0.0008) -[2023-10-15 16:39:31,121][52866] Updated weights for policy 1, policy_version 46780 (0.0009) -[2023-10-15 16:39:32,028][52833] Updated weights for policy 0, policy_version 46630 (0.0010) -[2023-10-15 16:39:32,395][52833] Updated weights for policy 0, policy_version 46640 (0.0007) -[2023-10-15 16:39:32,764][52833] Updated weights for policy 0, policy_version 46650 (0.0008) -[2023-10-15 16:39:33,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 95682560. Throughput: 0: 1772.1, 1: 1816.2. Samples: 23921424. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 16:39:33,441][51532] Avg episode reward: [(0, '57.210'), (1, '44.950')] -[2023-10-15 16:39:33,442][52410] Saving new best policy, reward=57.210! -[2023-10-15 16:39:34,866][52866] Updated weights for policy 1, policy_version 46790 (0.0008) -[2023-10-15 16:39:35,235][52866] Updated weights for policy 1, policy_version 46800 (0.0010) -[2023-10-15 16:39:35,590][52866] Updated weights for policy 1, policy_version 46810 (0.0011) -[2023-10-15 16:39:36,585][52833] Updated weights for policy 0, policy_version 46660 (0.0008) -[2023-10-15 16:39:36,957][52833] Updated weights for policy 0, policy_version 46670 (0.0011) -[2023-10-15 16:39:37,321][52833] Updated weights for policy 0, policy_version 46680 (0.0011) -[2023-10-15 16:39:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 95748096. Throughput: 0: 1799.3, 1: 1812.4. Samples: 23943080. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 16:39:38,442][51532] Avg episode reward: [(0, '56.200'), (1, '44.570')] -[2023-10-15 16:39:39,428][52866] Updated weights for policy 1, policy_version 46820 (0.0008) -[2023-10-15 16:39:39,799][52866] Updated weights for policy 1, policy_version 46830 (0.0010) -[2023-10-15 16:39:40,160][52866] Updated weights for policy 1, policy_version 46840 (0.0007) -[2023-10-15 16:39:41,160][52833] Updated weights for policy 0, policy_version 46690 (0.0011) -[2023-10-15 16:39:41,521][52833] Updated weights for policy 0, policy_version 46700 (0.0011) -[2023-10-15 16:39:41,893][52833] Updated weights for policy 0, policy_version 46710 (0.0010) -[2023-10-15 16:39:42,269][52833] Updated weights for policy 0, policy_version 46720 (0.0010) -[2023-10-15 16:39:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95813632. Throughput: 0: 1784.0, 1: 1809.6. Samples: 23964816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:39:43,442][51532] Avg episode reward: [(0, '56.670'), (1, '45.580')] -[2023-10-15 16:39:43,453][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000046720_47841280.pth... -[2023-10-15 16:39:43,453][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000046848_47972352.pth... -[2023-10-15 16:39:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000045184_46268416.pth -[2023-10-15 16:39:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000045056_46137344.pth -[2023-10-15 16:39:43,779][52866] Updated weights for policy 1, policy_version 46850 (0.0009) -[2023-10-15 16:39:44,149][52866] Updated weights for policy 1, policy_version 46860 (0.0008) -[2023-10-15 16:39:44,519][52866] Updated weights for policy 1, policy_version 46870 (0.0008) -[2023-10-15 16:39:44,879][52866] Updated weights for policy 1, policy_version 46880 (0.0009) -[2023-10-15 16:39:46,053][52833] Updated weights for policy 0, policy_version 46730 (0.0007) -[2023-10-15 16:39:46,425][52833] Updated weights for policy 0, policy_version 46740 (0.0007) -[2023-10-15 16:39:46,792][52833] Updated weights for policy 0, policy_version 46750 (0.0008) -[2023-10-15 16:39:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 95879168. Throughput: 0: 1806.0, 1: 1811.7. Samples: 23976032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:39:48,442][51532] Avg episode reward: [(0, '55.960'), (1, '46.100')] -[2023-10-15 16:39:48,593][52866] Updated weights for policy 1, policy_version 46890 (0.0008) -[2023-10-15 16:39:48,961][52866] Updated weights for policy 1, policy_version 46900 (0.0007) -[2023-10-15 16:39:49,329][52866] Updated weights for policy 1, policy_version 46910 (0.0008) -[2023-10-15 16:39:50,663][52833] Updated weights for policy 0, policy_version 46760 (0.0009) -[2023-10-15 16:39:51,028][52833] Updated weights for policy 0, policy_version 46770 (0.0009) -[2023-10-15 16:39:51,403][52833] Updated weights for policy 0, policy_version 46780 (0.0007) -[2023-10-15 16:39:52,968][52866] Updated weights for policy 1, policy_version 46920 (0.0007) -[2023-10-15 16:39:53,340][52866] Updated weights for policy 1, policy_version 46930 (0.0007) -[2023-10-15 16:39:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 95944704. Throughput: 0: 1784.1, 1: 1807.6. Samples: 23997166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:39:53,441][51532] Avg episode reward: [(0, '55.000'), (1, '44.440')] -[2023-10-15 16:39:53,717][52866] Updated weights for policy 1, policy_version 46940 (0.0007) -[2023-10-15 16:39:55,016][52833] Updated weights for policy 0, policy_version 46790 (0.0009) -[2023-10-15 16:39:55,387][52833] Updated weights for policy 0, policy_version 46800 (0.0011) -[2023-10-15 16:39:55,752][52833] Updated weights for policy 0, policy_version 46810 (0.0010) -[2023-10-15 16:39:57,378][52866] Updated weights for policy 1, policy_version 46950 (0.0009) -[2023-10-15 16:39:57,743][52866] Updated weights for policy 1, policy_version 46960 (0.0008) -[2023-10-15 16:39:58,112][52866] Updated weights for policy 1, policy_version 46970 (0.0009) -[2023-10-15 16:39:58,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 96043008. Throughput: 0: 1783.3, 1: 1813.8. Samples: 24018814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:39:58,442][51532] Avg episode reward: [(0, '56.100'), (1, '45.990')] -[2023-10-15 16:39:59,458][52833] Updated weights for policy 0, policy_version 46820 (0.0009) -[2023-10-15 16:39:59,829][52833] Updated weights for policy 0, policy_version 46830 (0.0012) -[2023-10-15 16:40:00,199][52833] Updated weights for policy 0, policy_version 46840 (0.0008) -[2023-10-15 16:40:01,942][52866] Updated weights for policy 1, policy_version 46980 (0.0008) -[2023-10-15 16:40:02,330][52866] Updated weights for policy 1, policy_version 46990 (0.0008) -[2023-10-15 16:40:02,706][52866] Updated weights for policy 1, policy_version 47000 (0.0007) -[2023-10-15 16:40:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 96108544. Throughput: 0: 1783.5, 1: 1807.6. Samples: 24029512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:03,441][51532] Avg episode reward: [(0, '56.010'), (1, '43.020')] -[2023-10-15 16:40:03,756][52833] Updated weights for policy 0, policy_version 46850 (0.0009) -[2023-10-15 16:40:04,121][52833] Updated weights for policy 0, policy_version 46860 (0.0011) -[2023-10-15 16:40:04,492][52833] Updated weights for policy 0, policy_version 46870 (0.0010) -[2023-10-15 16:40:04,855][52833] Updated weights for policy 0, policy_version 46880 (0.0008) -[2023-10-15 16:40:06,578][52866] Updated weights for policy 1, policy_version 47010 (0.0009) -[2023-10-15 16:40:06,943][52866] Updated weights for policy 1, policy_version 47020 (0.0007) -[2023-10-15 16:40:07,307][52866] Updated weights for policy 1, policy_version 47030 (0.0009) -[2023-10-15 16:40:07,673][52866] Updated weights for policy 1, policy_version 47040 (0.0008) -[2023-10-15 16:40:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 96174080. Throughput: 0: 1790.1, 1: 1814.1. Samples: 24051220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:08,442][51532] Avg episode reward: [(0, '56.720'), (1, '43.040')] -[2023-10-15 16:40:08,508][52833] Updated weights for policy 0, policy_version 46890 (0.0007) -[2023-10-15 16:40:08,875][52833] Updated weights for policy 0, policy_version 46900 (0.0009) -[2023-10-15 16:40:09,237][52833] Updated weights for policy 0, policy_version 46910 (0.0007) -[2023-10-15 16:40:11,457][52866] Updated weights for policy 1, policy_version 47050 (0.0009) -[2023-10-15 16:40:11,838][52866] Updated weights for policy 1, policy_version 47060 (0.0010) -[2023-10-15 16:40:12,203][52866] Updated weights for policy 1, policy_version 47070 (0.0011) -[2023-10-15 16:40:12,870][52833] Updated weights for policy 0, policy_version 46920 (0.0007) -[2023-10-15 16:40:13,235][52833] Updated weights for policy 0, policy_version 46930 (0.0007) -[2023-10-15 16:40:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 96239616. Throughput: 0: 1805.4, 1: 1806.3. Samples: 24072842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:13,442][51532] Avg episode reward: [(0, '53.370'), (1, '41.850')] -[2023-10-15 16:40:13,611][52833] Updated weights for policy 0, policy_version 46940 (0.0009) -[2023-10-15 16:40:15,998][52866] Updated weights for policy 1, policy_version 47080 (0.0007) -[2023-10-15 16:40:16,368][52866] Updated weights for policy 1, policy_version 47090 (0.0007) -[2023-10-15 16:40:16,731][52866] Updated weights for policy 1, policy_version 47100 (0.0007) -[2023-10-15 16:40:17,490][52833] Updated weights for policy 0, policy_version 46950 (0.0009) -[2023-10-15 16:40:17,880][52833] Updated weights for policy 0, policy_version 46960 (0.0008) -[2023-10-15 16:40:18,249][52833] Updated weights for policy 0, policy_version 46970 (0.0007) -[2023-10-15 16:40:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 96305152. Throughput: 0: 1796.4, 1: 1818.5. Samples: 24084096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:18,442][51532] Avg episode reward: [(0, '52.610'), (1, '44.780')] -[2023-10-15 16:40:20,427][52866] Updated weights for policy 1, policy_version 47110 (0.0008) -[2023-10-15 16:40:20,794][52866] Updated weights for policy 1, policy_version 47120 (0.0007) -[2023-10-15 16:40:21,157][52866] Updated weights for policy 1, policy_version 47130 (0.0007) -[2023-10-15 16:40:22,145][52833] Updated weights for policy 0, policy_version 46980 (0.0009) -[2023-10-15 16:40:22,523][52833] Updated weights for policy 0, policy_version 46990 (0.0008) -[2023-10-15 16:40:22,895][52833] Updated weights for policy 0, policy_version 47000 (0.0010) -[2023-10-15 16:40:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 96403456. Throughput: 0: 1803.6, 1: 1801.5. Samples: 24105308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:23,442][51532] Avg episode reward: [(0, '52.890'), (1, '43.050')] -[2023-10-15 16:40:24,862][52866] Updated weights for policy 1, policy_version 47140 (0.0009) -[2023-10-15 16:40:25,222][52866] Updated weights for policy 1, policy_version 47150 (0.0011) -[2023-10-15 16:40:25,583][52866] Updated weights for policy 1, policy_version 47160 (0.0010) -[2023-10-15 16:40:26,649][52833] Updated weights for policy 0, policy_version 47010 (0.0008) -[2023-10-15 16:40:27,027][52833] Updated weights for policy 0, policy_version 47020 (0.0007) -[2023-10-15 16:40:27,401][52833] Updated weights for policy 0, policy_version 47030 (0.0007) -[2023-10-15 16:40:27,768][52833] Updated weights for policy 0, policy_version 47040 (0.0007) -[2023-10-15 16:40:28,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96468992. Throughput: 0: 1791.5, 1: 1793.1. Samples: 24126122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:28,442][51532] Avg episode reward: [(0, '53.880'), (1, '40.520')] -[2023-10-15 16:40:29,387][52866] Updated weights for policy 1, policy_version 47170 (0.0010) -[2023-10-15 16:40:29,757][52866] Updated weights for policy 1, policy_version 47180 (0.0008) -[2023-10-15 16:40:30,118][52866] Updated weights for policy 1, policy_version 47190 (0.0008) -[2023-10-15 16:40:30,485][52866] Updated weights for policy 1, policy_version 47200 (0.0009) -[2023-10-15 16:40:31,481][52833] Updated weights for policy 0, policy_version 47050 (0.0011) -[2023-10-15 16:40:31,843][52833] Updated weights for policy 0, policy_version 47060 (0.0009) -[2023-10-15 16:40:32,215][52833] Updated weights for policy 0, policy_version 47070 (0.0009) -[2023-10-15 16:40:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 96534528. Throughput: 0: 1799.3, 1: 1786.1. Samples: 24137374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:33,442][51532] Avg episode reward: [(0, '52.430'), (1, '39.750')] -[2023-10-15 16:40:34,229][52866] Updated weights for policy 1, policy_version 47210 (0.0009) -[2023-10-15 16:40:34,595][52866] Updated weights for policy 1, policy_version 47220 (0.0008) -[2023-10-15 16:40:34,963][52866] Updated weights for policy 1, policy_version 47230 (0.0010) -[2023-10-15 16:40:36,036][52833] Updated weights for policy 0, policy_version 47080 (0.0011) -[2023-10-15 16:40:36,400][52833] Updated weights for policy 0, policy_version 47090 (0.0009) -[2023-10-15 16:40:36,766][52833] Updated weights for policy 0, policy_version 47100 (0.0009) -[2023-10-15 16:40:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 96600064. Throughput: 0: 1794.9, 1: 1785.2. Samples: 24158268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:38,441][51532] Avg episode reward: [(0, '51.590'), (1, '43.170')] -[2023-10-15 16:40:38,764][52866] Updated weights for policy 1, policy_version 47240 (0.0008) -[2023-10-15 16:40:39,125][52866] Updated weights for policy 1, policy_version 47250 (0.0008) -[2023-10-15 16:40:39,497][52866] Updated weights for policy 1, policy_version 47260 (0.0008) -[2023-10-15 16:40:40,590][52833] Updated weights for policy 0, policy_version 47110 (0.0010) -[2023-10-15 16:40:40,961][52833] Updated weights for policy 0, policy_version 47120 (0.0011) -[2023-10-15 16:40:41,330][52833] Updated weights for policy 0, policy_version 47130 (0.0011) -[2023-10-15 16:40:43,022][52866] Updated weights for policy 1, policy_version 47270 (0.0007) -[2023-10-15 16:40:43,392][52866] Updated weights for policy 1, policy_version 47280 (0.0008) -[2023-10-15 16:40:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.2). Total num frames: 96665600. Throughput: 0: 1785.9, 1: 1807.5. Samples: 24180516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:43,442][51532] Avg episode reward: [(0, '54.540'), (1, '41.940')] -[2023-10-15 16:40:43,763][52866] Updated weights for policy 1, policy_version 47290 (0.0007) -[2023-10-15 16:40:44,971][52833] Updated weights for policy 0, policy_version 47140 (0.0008) -[2023-10-15 16:40:45,345][52833] Updated weights for policy 0, policy_version 47150 (0.0008) -[2023-10-15 16:40:45,714][52833] Updated weights for policy 0, policy_version 47160 (0.0009) -[2023-10-15 16:40:47,590][52866] Updated weights for policy 1, policy_version 47300 (0.0007) -[2023-10-15 16:40:47,976][52866] Updated weights for policy 1, policy_version 47310 (0.0008) -[2023-10-15 16:40:48,341][52866] Updated weights for policy 1, policy_version 47320 (0.0009) -[2023-10-15 16:40:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 96731136. Throughput: 0: 1795.4, 1: 1788.3. Samples: 24190780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:48,441][51532] Avg episode reward: [(0, '50.040'), (1, '42.450')] -[2023-10-15 16:40:49,605][52833] Updated weights for policy 0, policy_version 47170 (0.0009) -[2023-10-15 16:40:49,978][52833] Updated weights for policy 0, policy_version 47180 (0.0010) -[2023-10-15 16:40:50,357][52833] Updated weights for policy 0, policy_version 47190 (0.0010) -[2023-10-15 16:40:50,714][52833] Updated weights for policy 0, policy_version 47200 (0.0011) -[2023-10-15 16:40:51,953][52866] Updated weights for policy 1, policy_version 47330 (0.0008) -[2023-10-15 16:40:52,317][52866] Updated weights for policy 1, policy_version 47340 (0.0010) -[2023-10-15 16:40:52,678][52866] Updated weights for policy 1, policy_version 47350 (0.0010) -[2023-10-15 16:40:53,036][52866] Updated weights for policy 1, policy_version 47360 (0.0008) -[2023-10-15 16:40:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 96829440. Throughput: 0: 1782.8, 1: 1808.7. Samples: 24212838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:53,442][51532] Avg episode reward: [(0, '50.610'), (1, '44.720')] -[2023-10-15 16:40:54,266][52833] Updated weights for policy 0, policy_version 47210 (0.0009) -[2023-10-15 16:40:54,638][52833] Updated weights for policy 0, policy_version 47220 (0.0008) -[2023-10-15 16:40:55,015][52833] Updated weights for policy 0, policy_version 47230 (0.0009) -[2023-10-15 16:40:56,810][52866] Updated weights for policy 1, policy_version 47370 (0.0010) -[2023-10-15 16:40:57,174][52866] Updated weights for policy 1, policy_version 47380 (0.0009) -[2023-10-15 16:40:57,547][52866] Updated weights for policy 1, policy_version 47390 (0.0009) -[2023-10-15 16:40:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 96894976. Throughput: 0: 1789.0, 1: 1793.3. Samples: 24234044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:40:58,441][51532] Avg episode reward: [(0, '52.160'), (1, '45.190')] -[2023-10-15 16:40:58,792][52833] Updated weights for policy 0, policy_version 47240 (0.0010) -[2023-10-15 16:40:59,170][52833] Updated weights for policy 0, policy_version 47250 (0.0009) -[2023-10-15 16:40:59,537][52833] Updated weights for policy 0, policy_version 47260 (0.0008) -[2023-10-15 16:41:01,208][52866] Updated weights for policy 1, policy_version 47400 (0.0010) -[2023-10-15 16:41:01,584][52866] Updated weights for policy 1, policy_version 47410 (0.0010) -[2023-10-15 16:41:01,948][52866] Updated weights for policy 1, policy_version 47420 (0.0010) -[2023-10-15 16:41:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 96960512. Throughput: 0: 1778.3, 1: 1800.7. Samples: 24245152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:41:03,442][51532] Avg episode reward: [(0, '52.690'), (1, '44.560')] -[2023-10-15 16:41:03,458][52833] Updated weights for policy 0, policy_version 47270 (0.0008) -[2023-10-15 16:41:03,833][52833] Updated weights for policy 0, policy_version 47280 (0.0008) -[2023-10-15 16:41:04,206][52833] Updated weights for policy 0, policy_version 47290 (0.0008) -[2023-10-15 16:41:05,640][52866] Updated weights for policy 1, policy_version 47430 (0.0012) -[2023-10-15 16:41:06,005][52866] Updated weights for policy 1, policy_version 47440 (0.0010) -[2023-10-15 16:41:06,369][52866] Updated weights for policy 1, policy_version 47450 (0.0007) -[2023-10-15 16:41:08,056][52833] Updated weights for policy 0, policy_version 47300 (0.0007) -[2023-10-15 16:41:08,433][52833] Updated weights for policy 0, policy_version 47310 (0.0007) -[2023-10-15 16:41:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 97026048. Throughput: 0: 1781.2, 1: 1792.6. Samples: 24266132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:41:08,442][51532] Avg episode reward: [(0, '51.430'), (1, '45.330')] -[2023-10-15 16:41:08,808][52833] Updated weights for policy 0, policy_version 47320 (0.0007) -[2023-10-15 16:41:10,292][52866] Updated weights for policy 1, policy_version 47460 (0.0009) -[2023-10-15 16:41:10,651][52866] Updated weights for policy 1, policy_version 47470 (0.0009) -[2023-10-15 16:41:11,021][52866] Updated weights for policy 1, policy_version 47480 (0.0009) -[2023-10-15 16:41:12,392][52833] Updated weights for policy 0, policy_version 47330 (0.0007) -[2023-10-15 16:41:12,772][52833] Updated weights for policy 0, policy_version 47340 (0.0009) -[2023-10-15 16:41:13,140][52833] Updated weights for policy 0, policy_version 47350 (0.0008) -[2023-10-15 16:41:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 97091584. Throughput: 0: 1800.1, 1: 1792.4. Samples: 24287786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:41:13,441][51532] Avg episode reward: [(0, '47.890'), (1, '46.290')] -[2023-10-15 16:41:13,506][52833] Updated weights for policy 0, policy_version 47360 (0.0008) -[2023-10-15 16:41:14,719][52866] Updated weights for policy 1, policy_version 47490 (0.0010) -[2023-10-15 16:41:15,086][52866] Updated weights for policy 1, policy_version 47500 (0.0008) -[2023-10-15 16:41:15,451][52866] Updated weights for policy 1, policy_version 47510 (0.0008) -[2023-10-15 16:41:15,817][52866] Updated weights for policy 1, policy_version 47520 (0.0008) -[2023-10-15 16:41:17,312][52833] Updated weights for policy 0, policy_version 47370 (0.0009) -[2023-10-15 16:41:17,682][52833] Updated weights for policy 0, policy_version 47380 (0.0009) -[2023-10-15 16:41:18,056][52833] Updated weights for policy 0, policy_version 47390 (0.0009) -[2023-10-15 16:41:18,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 97189888. Throughput: 0: 1779.7, 1: 1795.1. Samples: 24298238. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) -[2023-10-15 16:41:18,442][51532] Avg episode reward: [(0, '49.180'), (1, '44.750')] -[2023-10-15 16:41:19,634][52866] Updated weights for policy 1, policy_version 47530 (0.0009) -[2023-10-15 16:41:20,006][52866] Updated weights for policy 1, policy_version 47540 (0.0011) -[2023-10-15 16:41:20,371][52866] Updated weights for policy 1, policy_version 47550 (0.0009) -[2023-10-15 16:41:21,797][52833] Updated weights for policy 0, policy_version 47400 (0.0008) -[2023-10-15 16:41:22,170][52833] Updated weights for policy 0, policy_version 47410 (0.0008) -[2023-10-15 16:41:22,545][52833] Updated weights for policy 0, policy_version 47420 (0.0009) -[2023-10-15 16:41:23,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 97255424. Throughput: 0: 1800.8, 1: 1802.7. Samples: 24320430. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) -[2023-10-15 16:41:23,442][51532] Avg episode reward: [(0, '50.240'), (1, '44.420')] -[2023-10-15 16:41:24,031][52866] Updated weights for policy 1, policy_version 47560 (0.0007) -[2023-10-15 16:41:24,409][52866] Updated weights for policy 1, policy_version 47570 (0.0011) -[2023-10-15 16:41:24,775][52866] Updated weights for policy 1, policy_version 47580 (0.0011) -[2023-10-15 16:41:26,214][52833] Updated weights for policy 0, policy_version 47430 (0.0010) -[2023-10-15 16:41:26,584][52833] Updated weights for policy 0, policy_version 47440 (0.0009) -[2023-10-15 16:41:26,966][52833] Updated weights for policy 0, policy_version 47450 (0.0011) -[2023-10-15 16:41:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97320960. Throughput: 0: 1783.6, 1: 1806.0. Samples: 24342048. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) -[2023-10-15 16:41:28,441][51532] Avg episode reward: [(0, '49.800'), (1, '44.190')] -[2023-10-15 16:41:28,574][52866] Updated weights for policy 1, policy_version 47590 (0.0009) -[2023-10-15 16:41:28,943][52866] Updated weights for policy 1, policy_version 47600 (0.0010) -[2023-10-15 16:41:29,302][52866] Updated weights for policy 1, policy_version 47610 (0.0010) -[2023-10-15 16:41:30,850][52833] Updated weights for policy 0, policy_version 47460 (0.0009) -[2023-10-15 16:41:31,220][52833] Updated weights for policy 0, policy_version 47470 (0.0008) -[2023-10-15 16:41:31,587][52833] Updated weights for policy 0, policy_version 47480 (0.0009) -[2023-10-15 16:41:33,074][52866] Updated weights for policy 1, policy_version 47620 (0.0010) -[2023-10-15 16:41:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 97386496. Throughput: 0: 1806.1, 1: 1802.7. Samples: 24353178. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) -[2023-10-15 16:41:33,442][51532] Avg episode reward: [(0, '51.060'), (1, '44.720')] -[2023-10-15 16:41:33,457][52866] Updated weights for policy 1, policy_version 47630 (0.0009) -[2023-10-15 16:41:33,810][52866] Updated weights for policy 1, policy_version 47640 (0.0009) -[2023-10-15 16:41:35,276][52833] Updated weights for policy 0, policy_version 47490 (0.0007) -[2023-10-15 16:41:35,650][52833] Updated weights for policy 0, policy_version 47500 (0.0008) -[2023-10-15 16:41:36,006][52833] Updated weights for policy 0, policy_version 47510 (0.0008) -[2023-10-15 16:41:36,380][52833] Updated weights for policy 0, policy_version 47520 (0.0011) -[2023-10-15 16:41:37,523][52866] Updated weights for policy 1, policy_version 47650 (0.0009) -[2023-10-15 16:41:37,887][52866] Updated weights for policy 1, policy_version 47660 (0.0012) -[2023-10-15 16:41:38,264][52866] Updated weights for policy 1, policy_version 47670 (0.0010) -[2023-10-15 16:41:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 97452032. Throughput: 0: 1786.2, 1: 1804.8. Samples: 24374436. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) -[2023-10-15 16:41:38,442][51532] Avg episode reward: [(0, '50.350'), (1, '46.860')] -[2023-10-15 16:41:38,629][52866] Updated weights for policy 1, policy_version 47680 (0.0008) -[2023-10-15 16:41:40,276][52833] Updated weights for policy 0, policy_version 47530 (0.0009) -[2023-10-15 16:41:40,643][52833] Updated weights for policy 0, policy_version 47540 (0.0008) -[2023-10-15 16:41:41,015][52833] Updated weights for policy 0, policy_version 47550 (0.0008) -[2023-10-15 16:41:42,314][52866] Updated weights for policy 1, policy_version 47690 (0.0011) -[2023-10-15 16:41:42,693][52866] Updated weights for policy 1, policy_version 47700 (0.0009) -[2023-10-15 16:41:43,063][52866] Updated weights for policy 1, policy_version 47710 (0.0010) -[2023-10-15 16:41:43,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 97550336. Throughput: 0: 1782.3, 1: 1808.8. Samples: 24395648. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) -[2023-10-15 16:41:43,442][51532] Avg episode reward: [(0, '46.750'), (1, '46.340')] -[2023-10-15 16:41:43,455][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000047712_48857088.pth... -[2023-10-15 16:41:43,456][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000047552_48693248.pth... -[2023-10-15 16:41:43,485][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000046016_47120384.pth -[2023-10-15 16:41:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000045888_46989312.pth -[2023-10-15 16:41:44,792][52833] Updated weights for policy 0, policy_version 47560 (0.0008) -[2023-10-15 16:41:45,166][52833] Updated weights for policy 0, policy_version 47570 (0.0009) -[2023-10-15 16:41:45,541][52833] Updated weights for policy 0, policy_version 47580 (0.0008) -[2023-10-15 16:41:46,899][52866] Updated weights for policy 1, policy_version 47720 (0.0010) -[2023-10-15 16:41:47,263][52866] Updated weights for policy 1, policy_version 47730 (0.0011) -[2023-10-15 16:41:47,628][52866] Updated weights for policy 1, policy_version 47740 (0.0011) -[2023-10-15 16:41:48,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 97615872. Throughput: 0: 1785.7, 1: 1802.2. Samples: 24406604. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) -[2023-10-15 16:41:48,441][51532] Avg episode reward: [(0, '47.790'), (1, '45.720')] -[2023-10-15 16:41:49,319][52833] Updated weights for policy 0, policy_version 47590 (0.0007) -[2023-10-15 16:41:49,687][52833] Updated weights for policy 0, policy_version 47600 (0.0009) -[2023-10-15 16:41:50,063][52833] Updated weights for policy 0, policy_version 47610 (0.0008) -[2023-10-15 16:41:51,539][52866] Updated weights for policy 1, policy_version 47750 (0.0009) -[2023-10-15 16:41:51,904][52866] Updated weights for policy 1, policy_version 47760 (0.0010) -[2023-10-15 16:41:52,265][52866] Updated weights for policy 1, policy_version 47770 (0.0010) -[2023-10-15 16:41:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 97681408. Throughput: 0: 1785.4, 1: 1811.6. Samples: 24428000. Policy #0 lag: (min: 19.0, avg: 32.0, max: 51.0) -[2023-10-15 16:41:53,442][51532] Avg episode reward: [(0, '46.860'), (1, '47.940')] -[2023-10-15 16:41:53,881][52833] Updated weights for policy 0, policy_version 47620 (0.0007) -[2023-10-15 16:41:54,277][52833] Updated weights for policy 0, policy_version 47630 (0.0008) -[2023-10-15 16:41:54,645][52833] Updated weights for policy 0, policy_version 47640 (0.0007) -[2023-10-15 16:41:56,016][52866] Updated weights for policy 1, policy_version 47780 (0.0010) -[2023-10-15 16:41:56,380][52866] Updated weights for policy 1, policy_version 47790 (0.0009) -[2023-10-15 16:41:56,753][52866] Updated weights for policy 1, policy_version 47800 (0.0009) -[2023-10-15 16:41:58,177][52833] Updated weights for policy 0, policy_version 47650 (0.0010) -[2023-10-15 16:41:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 97746944. Throughput: 0: 1800.2, 1: 1800.1. Samples: 24449802. Policy #0 lag: (min: 19.0, avg: 32.0, max: 51.0) -[2023-10-15 16:41:58,442][51532] Avg episode reward: [(0, '46.600'), (1, '50.250')] -[2023-10-15 16:41:58,561][52833] Updated weights for policy 0, policy_version 47660 (0.0010) -[2023-10-15 16:41:58,918][52833] Updated weights for policy 0, policy_version 47670 (0.0010) -[2023-10-15 16:41:59,279][52833] Updated weights for policy 0, policy_version 47680 (0.0010) -[2023-10-15 16:42:00,201][52866] Updated weights for policy 1, policy_version 47810 (0.0009) -[2023-10-15 16:42:00,573][52866] Updated weights for policy 1, policy_version 47820 (0.0008) -[2023-10-15 16:42:00,949][52866] Updated weights for policy 1, policy_version 47830 (0.0009) -[2023-10-15 16:42:01,314][52866] Updated weights for policy 1, policy_version 47840 (0.0008) -[2023-10-15 16:42:03,139][52833] Updated weights for policy 0, policy_version 47690 (0.0008) -[2023-10-15 16:42:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 97812480. Throughput: 0: 1783.2, 1: 1813.8. Samples: 24460100. Policy #0 lag: (min: 19.0, avg: 32.0, max: 51.0) -[2023-10-15 16:42:03,442][51532] Avg episode reward: [(0, '45.840'), (1, '52.660')] -[2023-10-15 16:42:03,508][52833] Updated weights for policy 0, policy_version 47700 (0.0008) -[2023-10-15 16:42:03,879][52833] Updated weights for policy 0, policy_version 47710 (0.0008) -[2023-10-15 16:42:05,128][52866] Updated weights for policy 1, policy_version 47850 (0.0008) -[2023-10-15 16:42:05,494][52866] Updated weights for policy 1, policy_version 47860 (0.0008) -[2023-10-15 16:42:05,866][52866] Updated weights for policy 1, policy_version 47870 (0.0007) -[2023-10-15 16:42:07,560][52833] Updated weights for policy 0, policy_version 47720 (0.0008) -[2023-10-15 16:42:07,934][52833] Updated weights for policy 0, policy_version 47730 (0.0008) -[2023-10-15 16:42:08,304][52833] Updated weights for policy 0, policy_version 47740 (0.0009) -[2023-10-15 16:42:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 97878016. Throughput: 0: 1795.2, 1: 1793.3. Samples: 24481910. Policy #0 lag: (min: 19.0, avg: 32.0, max: 51.0) -[2023-10-15 16:42:08,442][51532] Avg episode reward: [(0, '44.650'), (1, '50.300')] -[2023-10-15 16:42:09,689][52866] Updated weights for policy 1, policy_version 47880 (0.0008) -[2023-10-15 16:42:10,065][52866] Updated weights for policy 1, policy_version 47890 (0.0007) -[2023-10-15 16:42:10,427][52866] Updated weights for policy 1, policy_version 47900 (0.0010) -[2023-10-15 16:42:12,054][52833] Updated weights for policy 0, policy_version 47750 (0.0007) -[2023-10-15 16:42:12,426][52833] Updated weights for policy 0, policy_version 47760 (0.0008) -[2023-10-15 16:42:12,800][52833] Updated weights for policy 0, policy_version 47770 (0.0010) -[2023-10-15 16:42:13,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 97976320. Throughput: 0: 1794.8, 1: 1788.8. Samples: 24503312. Policy #0 lag: (min: 19.0, avg: 32.0, max: 51.0) -[2023-10-15 16:42:13,442][51532] Avg episode reward: [(0, '44.810'), (1, '51.340')] -[2023-10-15 16:42:14,222][52866] Updated weights for policy 1, policy_version 47910 (0.0010) -[2023-10-15 16:42:14,582][52866] Updated weights for policy 1, policy_version 47920 (0.0011) -[2023-10-15 16:42:14,945][52866] Updated weights for policy 1, policy_version 47930 (0.0010) -[2023-10-15 16:42:16,470][52833] Updated weights for policy 0, policy_version 47780 (0.0008) -[2023-10-15 16:42:16,841][52833] Updated weights for policy 0, policy_version 47790 (0.0010) -[2023-10-15 16:42:17,216][52833] Updated weights for policy 0, policy_version 47800 (0.0009) -[2023-10-15 16:42:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98041856. Throughput: 0: 1797.0, 1: 1787.6. Samples: 24514482. Policy #0 lag: (min: 19.0, avg: 32.0, max: 51.0) -[2023-10-15 16:42:18,442][51532] Avg episode reward: [(0, '43.610'), (1, '51.780')] -[2023-10-15 16:42:18,733][52866] Updated weights for policy 1, policy_version 47940 (0.0008) -[2023-10-15 16:42:19,095][52866] Updated weights for policy 1, policy_version 47950 (0.0010) -[2023-10-15 16:42:19,459][52866] Updated weights for policy 1, policy_version 47960 (0.0007) -[2023-10-15 16:42:21,020][52833] Updated weights for policy 0, policy_version 47810 (0.0009) -[2023-10-15 16:42:21,391][52833] Updated weights for policy 0, policy_version 47820 (0.0008) -[2023-10-15 16:42:21,764][52833] Updated weights for policy 0, policy_version 47830 (0.0008) -[2023-10-15 16:42:22,137][52833] Updated weights for policy 0, policy_version 47840 (0.0007) -[2023-10-15 16:42:23,067][52866] Updated weights for policy 1, policy_version 47970 (0.0008) -[2023-10-15 16:42:23,424][52866] Updated weights for policy 1, policy_version 47980 (0.0008) -[2023-10-15 16:42:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98107392. Throughput: 0: 1800.9, 1: 1787.9. Samples: 24535930. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-15 16:42:23,442][51532] Avg episode reward: [(0, '45.540'), (1, '52.870')] -[2023-10-15 16:42:23,789][52866] Updated weights for policy 1, policy_version 47990 (0.0008) -[2023-10-15 16:42:24,157][52866] Updated weights for policy 1, policy_version 48000 (0.0008) -[2023-10-15 16:42:25,838][52833] Updated weights for policy 0, policy_version 47850 (0.0009) -[2023-10-15 16:42:26,221][52833] Updated weights for policy 0, policy_version 47860 (0.0009) -[2023-10-15 16:42:26,583][52833] Updated weights for policy 0, policy_version 47870 (0.0010) -[2023-10-15 16:42:27,819][52866] Updated weights for policy 1, policy_version 48010 (0.0007) -[2023-10-15 16:42:28,188][52866] Updated weights for policy 1, policy_version 48020 (0.0008) -[2023-10-15 16:42:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 98172928. Throughput: 0: 1792.3, 1: 1808.5. Samples: 24557686. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-15 16:42:28,442][51532] Avg episode reward: [(0, '43.610'), (1, '53.870')] -[2023-10-15 16:42:28,559][52866] Updated weights for policy 1, policy_version 48030 (0.0007) -[2023-10-15 16:42:28,628][52518] Saving new best policy, reward=53.870! -[2023-10-15 16:42:30,199][52833] Updated weights for policy 0, policy_version 47880 (0.0008) -[2023-10-15 16:42:30,574][52833] Updated weights for policy 0, policy_version 47890 (0.0007) -[2023-10-15 16:42:30,943][52833] Updated weights for policy 0, policy_version 47900 (0.0008) -[2023-10-15 16:42:32,181][52866] Updated weights for policy 1, policy_version 48040 (0.0009) -[2023-10-15 16:42:32,552][52866] Updated weights for policy 1, policy_version 48050 (0.0007) -[2023-10-15 16:42:32,924][52866] Updated weights for policy 1, policy_version 48060 (0.0008) -[2023-10-15 16:42:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 98271232. Throughput: 0: 1800.6, 1: 1803.0. Samples: 24568766. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-15 16:42:33,442][51532] Avg episode reward: [(0, '44.580'), (1, '52.310')] -[2023-10-15 16:42:34,791][52833] Updated weights for policy 0, policy_version 47910 (0.0008) -[2023-10-15 16:42:35,160][52833] Updated weights for policy 0, policy_version 47920 (0.0007) -[2023-10-15 16:42:35,526][52833] Updated weights for policy 0, policy_version 47930 (0.0010) -[2023-10-15 16:42:36,781][52866] Updated weights for policy 1, policy_version 48070 (0.0008) -[2023-10-15 16:42:37,151][52866] Updated weights for policy 1, policy_version 48080 (0.0007) -[2023-10-15 16:42:37,524][52866] Updated weights for policy 1, policy_version 48090 (0.0008) -[2023-10-15 16:42:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 98336768. Throughput: 0: 1791.9, 1: 1811.9. Samples: 24590170. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-15 16:42:38,442][51532] Avg episode reward: [(0, '42.630'), (1, '53.610')] -[2023-10-15 16:42:39,455][52833] Updated weights for policy 0, policy_version 47940 (0.0009) -[2023-10-15 16:42:39,846][52833] Updated weights for policy 0, policy_version 47950 (0.0011) -[2023-10-15 16:42:40,227][52833] Updated weights for policy 0, policy_version 47960 (0.0008) -[2023-10-15 16:42:41,360][52866] Updated weights for policy 1, policy_version 48100 (0.0010) -[2023-10-15 16:42:41,726][52866] Updated weights for policy 1, policy_version 48110 (0.0011) -[2023-10-15 16:42:42,102][52866] Updated weights for policy 1, policy_version 48120 (0.0011) -[2023-10-15 16:42:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14329.1). Total num frames: 98402304. Throughput: 0: 1791.4, 1: 1799.5. Samples: 24611392. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-15 16:42:43,441][51532] Avg episode reward: [(0, '40.590'), (1, '53.770')] -[2023-10-15 16:42:43,797][52833] Updated weights for policy 0, policy_version 47970 (0.0010) -[2023-10-15 16:42:44,165][52833] Updated weights for policy 0, policy_version 47980 (0.0011) -[2023-10-15 16:42:44,531][52833] Updated weights for policy 0, policy_version 47990 (0.0010) -[2023-10-15 16:42:44,896][52833] Updated weights for policy 0, policy_version 48000 (0.0009) -[2023-10-15 16:42:45,997][52866] Updated weights for policy 1, policy_version 48130 (0.0010) -[2023-10-15 16:42:46,359][52866] Updated weights for policy 1, policy_version 48140 (0.0010) -[2023-10-15 16:42:46,732][52866] Updated weights for policy 1, policy_version 48150 (0.0010) -[2023-10-15 16:42:47,088][52866] Updated weights for policy 1, policy_version 48160 (0.0008) -[2023-10-15 16:42:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 98467840. Throughput: 0: 1796.8, 1: 1814.3. Samples: 24622600. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-15 16:42:48,441][51532] Avg episode reward: [(0, '41.810'), (1, '52.490')] -[2023-10-15 16:42:48,673][52833] Updated weights for policy 0, policy_version 48010 (0.0009) -[2023-10-15 16:42:49,035][52833] Updated weights for policy 0, policy_version 48020 (0.0009) -[2023-10-15 16:42:49,406][52833] Updated weights for policy 0, policy_version 48030 (0.0009) -[2023-10-15 16:42:50,748][52866] Updated weights for policy 1, policy_version 48170 (0.0008) -[2023-10-15 16:42:51,109][52866] Updated weights for policy 1, policy_version 48180 (0.0011) -[2023-10-15 16:42:51,475][52866] Updated weights for policy 1, policy_version 48190 (0.0011) -[2023-10-15 16:42:53,097][52833] Updated weights for policy 0, policy_version 48040 (0.0008) -[2023-10-15 16:42:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 98533376. Throughput: 0: 1798.4, 1: 1801.4. Samples: 24643900. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-15 16:42:53,441][51532] Avg episode reward: [(0, '40.450'), (1, '48.770')] -[2023-10-15 16:42:53,455][52833] Updated weights for policy 0, policy_version 48050 (0.0010) -[2023-10-15 16:42:53,834][52833] Updated weights for policy 0, policy_version 48060 (0.0010) -[2023-10-15 16:42:55,139][52866] Updated weights for policy 1, policy_version 48200 (0.0009) -[2023-10-15 16:42:55,509][52866] Updated weights for policy 1, policy_version 48210 (0.0007) -[2023-10-15 16:42:55,869][52866] Updated weights for policy 1, policy_version 48220 (0.0008) -[2023-10-15 16:42:57,624][52833] Updated weights for policy 0, policy_version 48070 (0.0007) -[2023-10-15 16:42:57,997][52833] Updated weights for policy 0, policy_version 48080 (0.0007) -[2023-10-15 16:42:58,371][52833] Updated weights for policy 0, policy_version 48090 (0.0007) -[2023-10-15 16:42:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 98598912. Throughput: 0: 1812.1, 1: 1799.1. Samples: 24665814. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) -[2023-10-15 16:42:58,441][51532] Avg episode reward: [(0, '43.620'), (1, '46.370')] -[2023-10-15 16:42:59,521][52866] Updated weights for policy 1, policy_version 48230 (0.0008) -[2023-10-15 16:42:59,886][52866] Updated weights for policy 1, policy_version 48240 (0.0007) -[2023-10-15 16:43:00,250][52866] Updated weights for policy 1, policy_version 48250 (0.0012) -[2023-10-15 16:43:02,075][52833] Updated weights for policy 0, policy_version 48100 (0.0009) -[2023-10-15 16:43:02,436][52833] Updated weights for policy 0, policy_version 48110 (0.0008) -[2023-10-15 16:43:02,809][52833] Updated weights for policy 0, policy_version 48120 (0.0007) -[2023-10-15 16:43:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 98697216. Throughput: 0: 1794.0, 1: 1803.2. Samples: 24676352. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) -[2023-10-15 16:43:03,441][51532] Avg episode reward: [(0, '43.550'), (1, '44.860')] -[2023-10-15 16:43:04,137][52866] Updated weights for policy 1, policy_version 48260 (0.0010) -[2023-10-15 16:43:04,519][52866] Updated weights for policy 1, policy_version 48270 (0.0008) -[2023-10-15 16:43:04,885][52866] Updated weights for policy 1, policy_version 48280 (0.0007) -[2023-10-15 16:43:06,590][52833] Updated weights for policy 0, policy_version 48130 (0.0008) -[2023-10-15 16:43:06,967][52833] Updated weights for policy 0, policy_version 48140 (0.0009) -[2023-10-15 16:43:07,332][52833] Updated weights for policy 0, policy_version 48150 (0.0007) -[2023-10-15 16:43:07,701][52833] Updated weights for policy 0, policy_version 48160 (0.0009) -[2023-10-15 16:43:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 98762752. Throughput: 0: 1808.7, 1: 1797.6. Samples: 24698214. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) -[2023-10-15 16:43:08,442][51532] Avg episode reward: [(0, '42.130'), (1, '45.160')] -[2023-10-15 16:43:08,628][52866] Updated weights for policy 1, policy_version 48290 (0.0008) -[2023-10-15 16:43:08,997][52866] Updated weights for policy 1, policy_version 48300 (0.0008) -[2023-10-15 16:43:09,357][52866] Updated weights for policy 1, policy_version 48310 (0.0009) -[2023-10-15 16:43:09,719][52866] Updated weights for policy 1, policy_version 48320 (0.0007) -[2023-10-15 16:43:11,377][52833] Updated weights for policy 0, policy_version 48170 (0.0008) -[2023-10-15 16:43:11,746][52833] Updated weights for policy 0, policy_version 48180 (0.0008) -[2023-10-15 16:43:12,116][52833] Updated weights for policy 0, policy_version 48190 (0.0008) -[2023-10-15 16:43:13,333][52866] Updated weights for policy 1, policy_version 48330 (0.0010) -[2023-10-15 16:43:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98828288. Throughput: 0: 1797.7, 1: 1813.4. Samples: 24720186. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) -[2023-10-15 16:43:13,441][51532] Avg episode reward: [(0, '46.430'), (1, '45.510')] -[2023-10-15 16:43:13,697][52866] Updated weights for policy 1, policy_version 48340 (0.0010) -[2023-10-15 16:43:14,067][52866] Updated weights for policy 1, policy_version 48350 (0.0010) -[2023-10-15 16:43:15,850][52833] Updated weights for policy 0, policy_version 48200 (0.0008) -[2023-10-15 16:43:16,216][52833] Updated weights for policy 0, policy_version 48210 (0.0007) -[2023-10-15 16:43:16,584][52833] Updated weights for policy 0, policy_version 48220 (0.0008) -[2023-10-15 16:43:17,910][52866] Updated weights for policy 1, policy_version 48360 (0.0009) -[2023-10-15 16:43:18,291][52866] Updated weights for policy 1, policy_version 48370 (0.0008) -[2023-10-15 16:43:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 98893824. Throughput: 0: 1813.6, 1: 1790.2. Samples: 24730936. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) -[2023-10-15 16:43:18,441][51532] Avg episode reward: [(0, '44.770'), (1, '44.050')] -[2023-10-15 16:43:18,658][52866] Updated weights for policy 1, policy_version 48380 (0.0009) -[2023-10-15 16:43:20,338][52833] Updated weights for policy 0, policy_version 48230 (0.0009) -[2023-10-15 16:43:20,706][52833] Updated weights for policy 0, policy_version 48240 (0.0008) -[2023-10-15 16:43:21,077][52833] Updated weights for policy 0, policy_version 48250 (0.0008) -[2023-10-15 16:43:22,455][52866] Updated weights for policy 1, policy_version 48390 (0.0008) -[2023-10-15 16:43:22,828][52866] Updated weights for policy 1, policy_version 48400 (0.0007) -[2023-10-15 16:43:23,189][52866] Updated weights for policy 1, policy_version 48410 (0.0007) -[2023-10-15 16:43:23,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 98992128. Throughput: 0: 1801.7, 1: 1803.0. Samples: 24752380. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) -[2023-10-15 16:43:23,442][51532] Avg episode reward: [(0, '48.980'), (1, '44.560')] -[2023-10-15 16:43:24,785][52833] Updated weights for policy 0, policy_version 48260 (0.0010) -[2023-10-15 16:43:25,176][52833] Updated weights for policy 0, policy_version 48270 (0.0008) -[2023-10-15 16:43:25,548][52833] Updated weights for policy 0, policy_version 48280 (0.0009) -[2023-10-15 16:43:26,797][52866] Updated weights for policy 1, policy_version 48420 (0.0007) -[2023-10-15 16:43:27,163][52866] Updated weights for policy 1, policy_version 48430 (0.0010) -[2023-10-15 16:43:27,522][52866] Updated weights for policy 1, policy_version 48440 (0.0009) -[2023-10-15 16:43:28,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 99057664. Throughput: 0: 1801.1, 1: 1797.8. Samples: 24773340. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 16:43:28,442][51532] Avg episode reward: [(0, '50.750'), (1, '41.630')] -[2023-10-15 16:43:29,382][52833] Updated weights for policy 0, policy_version 48290 (0.0007) -[2023-10-15 16:43:29,764][52833] Updated weights for policy 0, policy_version 48300 (0.0009) -[2023-10-15 16:43:30,135][52833] Updated weights for policy 0, policy_version 48310 (0.0010) -[2023-10-15 16:43:30,496][52833] Updated weights for policy 0, policy_version 48320 (0.0007) -[2023-10-15 16:43:31,228][52866] Updated weights for policy 1, policy_version 48450 (0.0008) -[2023-10-15 16:43:31,601][52866] Updated weights for policy 1, policy_version 48460 (0.0009) -[2023-10-15 16:43:31,968][52866] Updated weights for policy 1, policy_version 48470 (0.0007) -[2023-10-15 16:43:32,337][52866] Updated weights for policy 1, policy_version 48480 (0.0007) -[2023-10-15 16:43:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 99123200. Throughput: 0: 1797.2, 1: 1803.1. Samples: 24784616. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 16:43:33,441][51532] Avg episode reward: [(0, '50.610'), (1, '41.990')] -[2023-10-15 16:43:34,273][52833] Updated weights for policy 0, policy_version 48330 (0.0009) -[2023-10-15 16:43:34,645][52833] Updated weights for policy 0, policy_version 48340 (0.0008) -[2023-10-15 16:43:35,018][52833] Updated weights for policy 0, policy_version 48350 (0.0007) -[2023-10-15 16:43:36,078][52866] Updated weights for policy 1, policy_version 48490 (0.0009) -[2023-10-15 16:43:36,449][52866] Updated weights for policy 1, policy_version 48500 (0.0008) -[2023-10-15 16:43:36,814][52866] Updated weights for policy 1, policy_version 48510 (0.0010) -[2023-10-15 16:43:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 99188736. Throughput: 0: 1795.5, 1: 1795.3. Samples: 24805484. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 16:43:38,442][51532] Avg episode reward: [(0, '50.560'), (1, '42.840')] -[2023-10-15 16:43:38,776][52833] Updated weights for policy 0, policy_version 48360 (0.0008) -[2023-10-15 16:43:39,144][52833] Updated weights for policy 0, policy_version 48370 (0.0008) -[2023-10-15 16:43:39,525][52833] Updated weights for policy 0, policy_version 48380 (0.0009) -[2023-10-15 16:43:40,506][52866] Updated weights for policy 1, policy_version 48520 (0.0010) -[2023-10-15 16:43:40,871][52866] Updated weights for policy 1, policy_version 48530 (0.0009) -[2023-10-15 16:43:41,233][52866] Updated weights for policy 1, policy_version 48540 (0.0009) -[2023-10-15 16:43:43,142][52833] Updated weights for policy 0, policy_version 48390 (0.0008) -[2023-10-15 16:43:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 99254272. Throughput: 0: 1806.3, 1: 1799.3. Samples: 24828068. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 16:43:43,442][51532] Avg episode reward: [(0, '53.610'), (1, '45.460')] -[2023-10-15 16:43:43,448][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000048544_49709056.pth... -[2023-10-15 16:43:43,485][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000046848_47972352.pth -[2023-10-15 16:43:43,515][52833] Updated weights for policy 0, policy_version 48400 (0.0009) -[2023-10-15 16:43:43,889][52833] Updated weights for policy 0, policy_version 48410 (0.0008) -[2023-10-15 16:43:44,106][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000048416_49577984.pth... -[2023-10-15 16:43:44,144][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000046720_47841280.pth -[2023-10-15 16:43:44,981][52866] Updated weights for policy 1, policy_version 48550 (0.0009) -[2023-10-15 16:43:45,339][52866] Updated weights for policy 1, policy_version 48560 (0.0008) -[2023-10-15 16:43:45,706][52866] Updated weights for policy 1, policy_version 48570 (0.0007) -[2023-10-15 16:43:47,617][52833] Updated weights for policy 0, policy_version 48420 (0.0008) -[2023-10-15 16:43:47,984][52833] Updated weights for policy 0, policy_version 48430 (0.0009) -[2023-10-15 16:43:48,352][52833] Updated weights for policy 0, policy_version 48440 (0.0010) -[2023-10-15 16:43:48,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 99319808. Throughput: 0: 1795.1, 1: 1799.2. Samples: 24838094. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 16:43:48,441][51532] Avg episode reward: [(0, '52.920'), (1, '48.010')] -[2023-10-15 16:43:49,488][52866] Updated weights for policy 1, policy_version 48580 (0.0007) -[2023-10-15 16:43:49,854][52866] Updated weights for policy 1, policy_version 48590 (0.0009) -[2023-10-15 16:43:50,214][52866] Updated weights for policy 1, policy_version 48600 (0.0008) -[2023-10-15 16:43:52,253][52833] Updated weights for policy 0, policy_version 48450 (0.0009) -[2023-10-15 16:43:52,616][52833] Updated weights for policy 0, policy_version 48460 (0.0010) -[2023-10-15 16:43:52,985][52833] Updated weights for policy 0, policy_version 48470 (0.0009) -[2023-10-15 16:43:53,355][52833] Updated weights for policy 0, policy_version 48480 (0.0008) -[2023-10-15 16:43:53,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 99418112. Throughput: 0: 1800.8, 1: 1803.2. Samples: 24860394. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 16:43:53,442][51532] Avg episode reward: [(0, '53.830'), (1, '48.160')] -[2023-10-15 16:43:53,895][52866] Updated weights for policy 1, policy_version 48610 (0.0010) -[2023-10-15 16:43:54,254][52866] Updated weights for policy 1, policy_version 48620 (0.0009) -[2023-10-15 16:43:54,633][52866] Updated weights for policy 1, policy_version 48630 (0.0008) -[2023-10-15 16:43:54,999][52866] Updated weights for policy 1, policy_version 48640 (0.0007) -[2023-10-15 16:43:57,192][52833] Updated weights for policy 0, policy_version 48490 (0.0010) -[2023-10-15 16:43:57,560][52833] Updated weights for policy 0, policy_version 48500 (0.0011) -[2023-10-15 16:43:57,929][52833] Updated weights for policy 0, policy_version 48510 (0.0009) -[2023-10-15 16:43:58,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 99483648. Throughput: 0: 1788.6, 1: 1796.6. Samples: 24881518. Policy #0 lag: (min: 21.0, avg: 21.2, max: 32.0) -[2023-10-15 16:43:58,441][51532] Avg episode reward: [(0, '54.760'), (1, '46.920')] -[2023-10-15 16:43:58,817][52866] Updated weights for policy 1, policy_version 48650 (0.0007) -[2023-10-15 16:43:59,180][52866] Updated weights for policy 1, policy_version 48660 (0.0009) -[2023-10-15 16:43:59,551][52866] Updated weights for policy 1, policy_version 48670 (0.0010) -[2023-10-15 16:44:01,625][52833] Updated weights for policy 0, policy_version 48520 (0.0010) -[2023-10-15 16:44:02,006][52833] Updated weights for policy 0, policy_version 48530 (0.0010) -[2023-10-15 16:44:02,377][52833] Updated weights for policy 0, policy_version 48540 (0.0010) -[2023-10-15 16:44:03,132][52866] Updated weights for policy 1, policy_version 48680 (0.0008) -[2023-10-15 16:44:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 99549184. Throughput: 0: 1794.1, 1: 1801.4. Samples: 24892736. Policy #0 lag: (min: 21.0, avg: 21.2, max: 32.0) -[2023-10-15 16:44:03,442][51532] Avg episode reward: [(0, '51.200'), (1, '48.280')] -[2023-10-15 16:44:03,505][52866] Updated weights for policy 1, policy_version 48690 (0.0009) -[2023-10-15 16:44:03,875][52866] Updated weights for policy 1, policy_version 48700 (0.0010) -[2023-10-15 16:44:06,120][52833] Updated weights for policy 0, policy_version 48550 (0.0009) -[2023-10-15 16:44:06,482][52833] Updated weights for policy 0, policy_version 48560 (0.0009) -[2023-10-15 16:44:06,853][52833] Updated weights for policy 0, policy_version 48570 (0.0010) -[2023-10-15 16:44:07,456][52866] Updated weights for policy 1, policy_version 48710 (0.0007) -[2023-10-15 16:44:07,821][52866] Updated weights for policy 1, policy_version 48720 (0.0008) -[2023-10-15 16:44:08,185][52866] Updated weights for policy 1, policy_version 48730 (0.0007) -[2023-10-15 16:44:08,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 99647488. Throughput: 0: 1789.1, 1: 1810.4. Samples: 24914356. Policy #0 lag: (min: 21.0, avg: 21.2, max: 32.0) -[2023-10-15 16:44:08,442][51532] Avg episode reward: [(0, '48.870'), (1, '49.250')] -[2023-10-15 16:44:10,637][52833] Updated weights for policy 0, policy_version 48580 (0.0010) -[2023-10-15 16:44:11,030][52833] Updated weights for policy 0, policy_version 48590 (0.0007) -[2023-10-15 16:44:11,401][52833] Updated weights for policy 0, policy_version 48600 (0.0009) -[2023-10-15 16:44:11,896][52866] Updated weights for policy 1, policy_version 48740 (0.0008) -[2023-10-15 16:44:12,257][52866] Updated weights for policy 1, policy_version 48750 (0.0007) -[2023-10-15 16:44:12,635][52866] Updated weights for policy 1, policy_version 48760 (0.0009) -[2023-10-15 16:44:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 99713024. Throughput: 0: 1779.4, 1: 1812.9. Samples: 24934992. Policy #0 lag: (min: 21.0, avg: 21.2, max: 32.0) -[2023-10-15 16:44:13,442][51532] Avg episode reward: [(0, '49.150'), (1, '50.180')] -[2023-10-15 16:44:15,024][52833] Updated weights for policy 0, policy_version 48610 (0.0009) -[2023-10-15 16:44:15,388][52833] Updated weights for policy 0, policy_version 48620 (0.0008) -[2023-10-15 16:44:15,753][52833] Updated weights for policy 0, policy_version 48630 (0.0008) -[2023-10-15 16:44:16,090][52866] Updated weights for policy 1, policy_version 48770 (0.0007) -[2023-10-15 16:44:16,123][52833] Updated weights for policy 0, policy_version 48640 (0.0009) -[2023-10-15 16:44:16,450][52866] Updated weights for policy 1, policy_version 48780 (0.0008) -[2023-10-15 16:44:16,826][52866] Updated weights for policy 1, policy_version 48790 (0.0009) -[2023-10-15 16:44:17,191][52866] Updated weights for policy 1, policy_version 48800 (0.0009) -[2023-10-15 16:44:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 99778560. Throughput: 0: 1788.1, 1: 1817.5. Samples: 24946868. Policy #0 lag: (min: 21.0, avg: 21.2, max: 32.0) -[2023-10-15 16:44:18,441][51532] Avg episode reward: [(0, '48.930'), (1, '49.880')] -[2023-10-15 16:44:19,813][52833] Updated weights for policy 0, policy_version 48650 (0.0010) -[2023-10-15 16:44:20,187][52833] Updated weights for policy 0, policy_version 48660 (0.0008) -[2023-10-15 16:44:20,549][52833] Updated weights for policy 0, policy_version 48670 (0.0010) -[2023-10-15 16:44:21,038][52866] Updated weights for policy 1, policy_version 48810 (0.0010) -[2023-10-15 16:44:21,403][52866] Updated weights for policy 1, policy_version 48820 (0.0010) -[2023-10-15 16:44:21,771][52866] Updated weights for policy 1, policy_version 48830 (0.0009) -[2023-10-15 16:44:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 99844096. Throughput: 0: 1782.2, 1: 1820.9. Samples: 24967622. Policy #0 lag: (min: 21.0, avg: 21.2, max: 32.0) -[2023-10-15 16:44:23,442][51532] Avg episode reward: [(0, '45.760'), (1, '50.520')] -[2023-10-15 16:44:24,350][52833] Updated weights for policy 0, policy_version 48680 (0.0008) -[2023-10-15 16:44:24,723][52833] Updated weights for policy 0, policy_version 48690 (0.0009) -[2023-10-15 16:44:25,092][52833] Updated weights for policy 0, policy_version 48700 (0.0010) -[2023-10-15 16:44:25,561][52866] Updated weights for policy 1, policy_version 48840 (0.0008) -[2023-10-15 16:44:25,930][52866] Updated weights for policy 1, policy_version 48850 (0.0008) -[2023-10-15 16:44:26,293][52866] Updated weights for policy 1, policy_version 48860 (0.0009) -[2023-10-15 16:44:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 99909632. Throughput: 0: 1781.9, 1: 1817.4. Samples: 24990038. Policy #0 lag: (min: 21.0, avg: 21.2, max: 32.0) -[2023-10-15 16:44:28,443][51532] Avg episode reward: [(0, '43.890'), (1, '46.450')] -[2023-10-15 16:44:28,914][52833] Updated weights for policy 0, policy_version 48710 (0.0009) -[2023-10-15 16:44:29,288][52833] Updated weights for policy 0, policy_version 48720 (0.0008) -[2023-10-15 16:44:29,661][52833] Updated weights for policy 0, policy_version 48730 (0.0008) -[2023-10-15 16:44:30,036][52866] Updated weights for policy 1, policy_version 48870 (0.0010) -[2023-10-15 16:44:30,392][52866] Updated weights for policy 1, policy_version 48880 (0.0008) -[2023-10-15 16:44:30,754][52866] Updated weights for policy 1, policy_version 48890 (0.0008) -[2023-10-15 16:44:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 99975168. Throughput: 0: 1781.3, 1: 1819.5. Samples: 25000128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:44:33,442][51532] Avg episode reward: [(0, '42.750'), (1, '47.490')] -[2023-10-15 16:44:33,506][52833] Updated weights for policy 0, policy_version 48740 (0.0008) -[2023-10-15 16:44:33,879][52833] Updated weights for policy 0, policy_version 48750 (0.0007) -[2023-10-15 16:44:34,246][52833] Updated weights for policy 0, policy_version 48760 (0.0011) -[2023-10-15 16:44:34,490][52866] Updated weights for policy 1, policy_version 48900 (0.0009) -[2023-10-15 16:44:34,854][52866] Updated weights for policy 1, policy_version 48910 (0.0009) -[2023-10-15 16:44:35,216][52866] Updated weights for policy 1, policy_version 48920 (0.0009) -[2023-10-15 16:44:38,160][52833] Updated weights for policy 0, policy_version 48770 (0.0008) -[2023-10-15 16:44:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 100040704. Throughput: 0: 1780.6, 1: 1810.4. Samples: 25021988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:44:38,442][51532] Avg episode reward: [(0, '41.020'), (1, '47.760')] -[2023-10-15 16:44:38,540][52833] Updated weights for policy 0, policy_version 48780 (0.0007) -[2023-10-15 16:44:38,906][52833] Updated weights for policy 0, policy_version 48790 (0.0007) -[2023-10-15 16:44:39,142][52866] Updated weights for policy 1, policy_version 48930 (0.0007) -[2023-10-15 16:44:39,271][52833] Updated weights for policy 0, policy_version 48800 (0.0007) -[2023-10-15 16:44:39,549][52866] Updated weights for policy 1, policy_version 48940 (0.0009) -[2023-10-15 16:44:39,905][52866] Updated weights for policy 1, policy_version 48950 (0.0007) -[2023-10-15 16:44:40,270][52866] Updated weights for policy 1, policy_version 48960 (0.0008) -[2023-10-15 16:44:42,946][52833] Updated weights for policy 0, policy_version 48810 (0.0007) -[2023-10-15 16:44:43,304][52833] Updated weights for policy 0, policy_version 48820 (0.0009) -[2023-10-15 16:44:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 100106240. Throughput: 0: 1806.0, 1: 1805.6. Samples: 25044044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:44:43,442][51532] Avg episode reward: [(0, '43.520'), (1, '45.240')] -[2023-10-15 16:44:43,682][52833] Updated weights for policy 0, policy_version 48830 (0.0010) -[2023-10-15 16:44:44,100][52866] Updated weights for policy 1, policy_version 48970 (0.0009) -[2023-10-15 16:44:44,469][52866] Updated weights for policy 1, policy_version 48980 (0.0010) -[2023-10-15 16:44:44,839][52866] Updated weights for policy 1, policy_version 48990 (0.0009) -[2023-10-15 16:44:47,345][52833] Updated weights for policy 0, policy_version 48840 (0.0009) -[2023-10-15 16:44:47,711][52833] Updated weights for policy 0, policy_version 48850 (0.0009) -[2023-10-15 16:44:48,083][52833] Updated weights for policy 0, policy_version 48860 (0.0009) -[2023-10-15 16:44:48,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 100204544. Throughput: 0: 1787.3, 1: 1806.4. Samples: 25054448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:44:48,441][51532] Avg episode reward: [(0, '44.200'), (1, '43.940')] -[2023-10-15 16:44:48,507][52866] Updated weights for policy 1, policy_version 49000 (0.0008) -[2023-10-15 16:44:48,876][52866] Updated weights for policy 1, policy_version 49010 (0.0007) -[2023-10-15 16:44:49,236][52866] Updated weights for policy 1, policy_version 49020 (0.0007) -[2023-10-15 16:44:51,961][52833] Updated weights for policy 0, policy_version 48870 (0.0009) -[2023-10-15 16:44:52,344][52833] Updated weights for policy 0, policy_version 48880 (0.0010) -[2023-10-15 16:44:52,712][52833] Updated weights for policy 0, policy_version 48890 (0.0009) -[2023-10-15 16:44:52,912][52866] Updated weights for policy 1, policy_version 49030 (0.0007) -[2023-10-15 16:44:53,274][52866] Updated weights for policy 1, policy_version 49040 (0.0010) -[2023-10-15 16:44:53,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 100270080. Throughput: 0: 1803.1, 1: 1800.5. Samples: 25076516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:44:53,441][51532] Avg episode reward: [(0, '45.800'), (1, '41.650')] -[2023-10-15 16:44:53,642][52866] Updated weights for policy 1, policy_version 49050 (0.0008) -[2023-10-15 16:44:56,543][52833] Updated weights for policy 0, policy_version 48900 (0.0008) -[2023-10-15 16:44:56,941][52833] Updated weights for policy 0, policy_version 48910 (0.0011) -[2023-10-15 16:44:57,318][52833] Updated weights for policy 0, policy_version 48920 (0.0008) -[2023-10-15 16:44:57,494][52866] Updated weights for policy 1, policy_version 49060 (0.0008) -[2023-10-15 16:44:57,857][52866] Updated weights for policy 1, policy_version 49070 (0.0008) -[2023-10-15 16:44:58,223][52866] Updated weights for policy 1, policy_version 49080 (0.0007) -[2023-10-15 16:44:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 100335616. Throughput: 0: 1784.7, 1: 1810.5. Samples: 25096774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:44:58,441][51532] Avg episode reward: [(0, '47.160'), (1, '40.640')] -[2023-10-15 16:45:00,977][52833] Updated weights for policy 0, policy_version 48930 (0.0009) -[2023-10-15 16:45:01,346][52833] Updated weights for policy 0, policy_version 48940 (0.0008) -[2023-10-15 16:45:01,713][52833] Updated weights for policy 0, policy_version 48950 (0.0007) -[2023-10-15 16:45:02,035][52866] Updated weights for policy 1, policy_version 49090 (0.0008) -[2023-10-15 16:45:02,078][52833] Updated weights for policy 0, policy_version 48960 (0.0007) -[2023-10-15 16:45:02,401][52866] Updated weights for policy 1, policy_version 49100 (0.0008) -[2023-10-15 16:45:02,762][52866] Updated weights for policy 1, policy_version 49110 (0.0009) -[2023-10-15 16:45:03,132][52866] Updated weights for policy 1, policy_version 49120 (0.0010) -[2023-10-15 16:45:03,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 100433920. Throughput: 0: 1815.8, 1: 1784.3. Samples: 25108872. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 16:45:03,442][51532] Avg episode reward: [(0, '46.280'), (1, '41.230')] -[2023-10-15 16:45:05,696][52833] Updated weights for policy 0, policy_version 48970 (0.0010) -[2023-10-15 16:45:06,064][52833] Updated weights for policy 0, policy_version 48980 (0.0008) -[2023-10-15 16:45:06,437][52833] Updated weights for policy 0, policy_version 48990 (0.0008) -[2023-10-15 16:45:06,869][52866] Updated weights for policy 1, policy_version 49130 (0.0008) -[2023-10-15 16:45:07,230][52866] Updated weights for policy 1, policy_version 49140 (0.0007) -[2023-10-15 16:45:07,601][52866] Updated weights for policy 1, policy_version 49150 (0.0009) -[2023-10-15 16:45:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100499456. Throughput: 0: 1787.9, 1: 1806.1. Samples: 25129352. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 16:45:08,442][51532] Avg episode reward: [(0, '45.220'), (1, '41.250')] -[2023-10-15 16:45:10,149][52833] Updated weights for policy 0, policy_version 49000 (0.0008) -[2023-10-15 16:45:10,525][52833] Updated weights for policy 0, policy_version 49010 (0.0011) -[2023-10-15 16:45:10,893][52833] Updated weights for policy 0, policy_version 49020 (0.0007) -[2023-10-15 16:45:11,223][52866] Updated weights for policy 1, policy_version 49160 (0.0008) -[2023-10-15 16:45:11,591][52866] Updated weights for policy 1, policy_version 49170 (0.0009) -[2023-10-15 16:45:11,955][52866] Updated weights for policy 1, policy_version 49180 (0.0012) -[2023-10-15 16:45:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100564992. Throughput: 0: 1789.0, 1: 1789.5. Samples: 25151068. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 16:45:13,442][51532] Avg episode reward: [(0, '49.240'), (1, '41.990')] -[2023-10-15 16:45:14,600][52833] Updated weights for policy 0, policy_version 49030 (0.0009) -[2023-10-15 16:45:14,970][52833] Updated weights for policy 0, policy_version 49040 (0.0008) -[2023-10-15 16:45:15,344][52833] Updated weights for policy 0, policy_version 49050 (0.0008) -[2023-10-15 16:45:15,612][52866] Updated weights for policy 1, policy_version 49190 (0.0009) -[2023-10-15 16:45:15,976][52866] Updated weights for policy 1, policy_version 49200 (0.0011) -[2023-10-15 16:45:16,348][52866] Updated weights for policy 1, policy_version 49210 (0.0010) -[2023-10-15 16:45:18,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 100630528. Throughput: 0: 1789.4, 1: 1804.4. Samples: 25161848. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 16:45:18,442][51532] Avg episode reward: [(0, '48.420'), (1, '42.790')] -[2023-10-15 16:45:18,960][52833] Updated weights for policy 0, policy_version 49060 (0.0007) -[2023-10-15 16:45:19,333][52833] Updated weights for policy 0, policy_version 49070 (0.0009) -[2023-10-15 16:45:19,691][52833] Updated weights for policy 0, policy_version 49080 (0.0010) -[2023-10-15 16:45:20,164][52866] Updated weights for policy 1, policy_version 49220 (0.0010) -[2023-10-15 16:45:20,527][52866] Updated weights for policy 1, policy_version 49230 (0.0008) -[2023-10-15 16:45:20,893][52866] Updated weights for policy 1, policy_version 49240 (0.0008) -[2023-10-15 16:45:23,378][52833] Updated weights for policy 0, policy_version 49090 (0.0008) -[2023-10-15 16:45:23,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 100696064. Throughput: 0: 1800.8, 1: 1793.7. Samples: 25183736. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 16:45:23,441][51532] Avg episode reward: [(0, '49.470'), (1, '44.000')] -[2023-10-15 16:45:23,752][52833] Updated weights for policy 0, policy_version 49100 (0.0009) -[2023-10-15 16:45:24,116][52833] Updated weights for policy 0, policy_version 49110 (0.0007) -[2023-10-15 16:45:24,495][52833] Updated weights for policy 0, policy_version 49120 (0.0008) -[2023-10-15 16:45:24,651][52866] Updated weights for policy 1, policy_version 49250 (0.0011) -[2023-10-15 16:45:25,061][52866] Updated weights for policy 1, policy_version 49260 (0.0009) -[2023-10-15 16:45:25,433][52866] Updated weights for policy 1, policy_version 49270 (0.0007) -[2023-10-15 16:45:25,803][52866] Updated weights for policy 1, policy_version 49280 (0.0007) -[2023-10-15 16:45:28,312][52833] Updated weights for policy 0, policy_version 49130 (0.0010) -[2023-10-15 16:45:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 100761600. Throughput: 0: 1808.1, 1: 1794.8. Samples: 25206172. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 16:45:28,442][51532] Avg episode reward: [(0, '49.880'), (1, '45.020')] -[2023-10-15 16:45:28,674][52833] Updated weights for policy 0, policy_version 49140 (0.0007) -[2023-10-15 16:45:29,046][52833] Updated weights for policy 0, policy_version 49150 (0.0007) -[2023-10-15 16:45:29,546][52866] Updated weights for policy 1, policy_version 49290 (0.0008) -[2023-10-15 16:45:29,916][52866] Updated weights for policy 1, policy_version 49300 (0.0009) -[2023-10-15 16:45:30,280][52866] Updated weights for policy 1, policy_version 49310 (0.0007) -[2023-10-15 16:45:32,864][52833] Updated weights for policy 0, policy_version 49160 (0.0007) -[2023-10-15 16:45:33,234][52833] Updated weights for policy 0, policy_version 49170 (0.0008) -[2023-10-15 16:45:33,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 100827136. Throughput: 0: 1797.5, 1: 1796.0. Samples: 25216154. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) -[2023-10-15 16:45:33,442][51532] Avg episode reward: [(0, '47.690'), (1, '46.380')] -[2023-10-15 16:45:33,610][52833] Updated weights for policy 0, policy_version 49180 (0.0009) -[2023-10-15 16:45:33,898][52866] Updated weights for policy 1, policy_version 49320 (0.0009) -[2023-10-15 16:45:34,263][52866] Updated weights for policy 1, policy_version 49330 (0.0009) -[2023-10-15 16:45:34,628][52866] Updated weights for policy 1, policy_version 49340 (0.0009) -[2023-10-15 16:45:37,461][52833] Updated weights for policy 0, policy_version 49190 (0.0008) -[2023-10-15 16:45:37,829][52833] Updated weights for policy 0, policy_version 49200 (0.0008) -[2023-10-15 16:45:38,198][52833] Updated weights for policy 0, policy_version 49210 (0.0008) -[2023-10-15 16:45:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 100925440. Throughput: 0: 1806.3, 1: 1792.7. Samples: 25238468. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:45:38,442][51532] Avg episode reward: [(0, '45.410'), (1, '47.290')] -[2023-10-15 16:45:38,508][52866] Updated weights for policy 1, policy_version 49350 (0.0009) -[2023-10-15 16:45:38,875][52866] Updated weights for policy 1, policy_version 49360 (0.0007) -[2023-10-15 16:45:39,247][52866] Updated weights for policy 1, policy_version 49370 (0.0008) -[2023-10-15 16:45:42,041][52833] Updated weights for policy 0, policy_version 49220 (0.0008) -[2023-10-15 16:45:42,423][52833] Updated weights for policy 0, policy_version 49230 (0.0009) -[2023-10-15 16:45:42,781][52833] Updated weights for policy 0, policy_version 49240 (0.0010) -[2023-10-15 16:45:42,920][52866] Updated weights for policy 1, policy_version 49380 (0.0009) -[2023-10-15 16:45:43,289][52866] Updated weights for policy 1, policy_version 49390 (0.0008) -[2023-10-15 16:45:43,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 100990976. Throughput: 0: 1806.4, 1: 1809.9. Samples: 25259510. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:45:43,441][51532] Avg episode reward: [(0, '46.800'), (1, '48.060')] -[2023-10-15 16:45:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000049248_50429952.pth... -[2023-10-15 16:45:43,484][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000047552_48693248.pth -[2023-10-15 16:45:43,663][52866] Updated weights for policy 1, policy_version 49400 (0.0011) -[2023-10-15 16:45:43,945][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000049408_50593792.pth... -[2023-10-15 16:45:43,974][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000047712_48857088.pth -[2023-10-15 16:45:46,492][52833] Updated weights for policy 0, policy_version 49250 (0.0009) -[2023-10-15 16:45:46,865][52833] Updated weights for policy 0, policy_version 49260 (0.0010) -[2023-10-15 16:45:47,229][52833] Updated weights for policy 0, policy_version 49270 (0.0010) -[2023-10-15 16:45:47,416][52866] Updated weights for policy 1, policy_version 49410 (0.0009) -[2023-10-15 16:45:47,597][52833] Updated weights for policy 0, policy_version 49280 (0.0008) -[2023-10-15 16:45:47,790][52866] Updated weights for policy 1, policy_version 49420 (0.0007) -[2023-10-15 16:45:48,150][52866] Updated weights for policy 1, policy_version 49430 (0.0007) -[2023-10-15 16:45:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 101056512. Throughput: 0: 1792.5, 1: 1804.1. Samples: 25270718. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:45:48,441][51532] Avg episode reward: [(0, '49.730'), (1, '48.890')] -[2023-10-15 16:45:48,516][52866] Updated weights for policy 1, policy_version 49440 (0.0009) -[2023-10-15 16:45:51,269][52833] Updated weights for policy 0, policy_version 49290 (0.0009) -[2023-10-15 16:45:51,640][52833] Updated weights for policy 0, policy_version 49300 (0.0008) -[2023-10-15 16:45:52,012][52833] Updated weights for policy 0, policy_version 49310 (0.0009) -[2023-10-15 16:45:52,182][52866] Updated weights for policy 1, policy_version 49450 (0.0007) -[2023-10-15 16:45:52,556][52866] Updated weights for policy 1, policy_version 49460 (0.0008) -[2023-10-15 16:45:52,912][52866] Updated weights for policy 1, policy_version 49470 (0.0008) -[2023-10-15 16:45:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 101154816. Throughput: 0: 1799.2, 1: 1813.0. Samples: 25291900. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:45:53,442][51532] Avg episode reward: [(0, '50.530'), (1, '48.870')] -[2023-10-15 16:45:55,786][52833] Updated weights for policy 0, policy_version 49320 (0.0007) -[2023-10-15 16:45:56,155][52833] Updated weights for policy 0, policy_version 49330 (0.0010) -[2023-10-15 16:45:56,524][52833] Updated weights for policy 0, policy_version 49340 (0.0008) -[2023-10-15 16:45:56,713][52866] Updated weights for policy 1, policy_version 49480 (0.0009) -[2023-10-15 16:45:57,081][52866] Updated weights for policy 1, policy_version 49490 (0.0009) -[2023-10-15 16:45:57,450][52866] Updated weights for policy 1, policy_version 49500 (0.0012) -[2023-10-15 16:45:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 101220352. Throughput: 0: 1786.2, 1: 1800.3. Samples: 25312462. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:45:58,442][51532] Avg episode reward: [(0, '50.150'), (1, '46.170')] -[2023-10-15 16:46:00,267][52833] Updated weights for policy 0, policy_version 49350 (0.0008) -[2023-10-15 16:46:00,641][52833] Updated weights for policy 0, policy_version 49360 (0.0010) -[2023-10-15 16:46:01,017][52833] Updated weights for policy 0, policy_version 49370 (0.0010) -[2023-10-15 16:46:01,166][52866] Updated weights for policy 1, policy_version 49510 (0.0009) -[2023-10-15 16:46:01,522][52866] Updated weights for policy 1, policy_version 49520 (0.0007) -[2023-10-15 16:46:01,890][52866] Updated weights for policy 1, policy_version 49530 (0.0011) -[2023-10-15 16:46:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101285888. Throughput: 0: 1796.0, 1: 1815.4. Samples: 25324358. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 16:46:03,442][51532] Avg episode reward: [(0, '49.250'), (1, '48.750')] -[2023-10-15 16:46:04,869][52833] Updated weights for policy 0, policy_version 49380 (0.0009) -[2023-10-15 16:46:05,249][52833] Updated weights for policy 0, policy_version 49390 (0.0011) -[2023-10-15 16:46:05,507][52866] Updated weights for policy 1, policy_version 49540 (0.0009) -[2023-10-15 16:46:05,613][52833] Updated weights for policy 0, policy_version 49400 (0.0008) -[2023-10-15 16:46:05,883][52866] Updated weights for policy 1, policy_version 49550 (0.0008) -[2023-10-15 16:46:06,244][52866] Updated weights for policy 1, policy_version 49560 (0.0008) -[2023-10-15 16:46:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101351424. Throughput: 0: 1773.6, 1: 1803.6. Samples: 25344712. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) -[2023-10-15 16:46:08,442][51532] Avg episode reward: [(0, '47.210'), (1, '46.040')] -[2023-10-15 16:46:09,425][52833] Updated weights for policy 0, policy_version 49410 (0.0009) -[2023-10-15 16:46:09,795][52833] Updated weights for policy 0, policy_version 49420 (0.0008) -[2023-10-15 16:46:10,018][52866] Updated weights for policy 1, policy_version 49570 (0.0008) -[2023-10-15 16:46:10,169][52833] Updated weights for policy 0, policy_version 49430 (0.0007) -[2023-10-15 16:46:10,421][52866] Updated weights for policy 1, policy_version 49580 (0.0007) -[2023-10-15 16:46:10,539][52833] Updated weights for policy 0, policy_version 49440 (0.0008) -[2023-10-15 16:46:10,787][52866] Updated weights for policy 1, policy_version 49590 (0.0009) -[2023-10-15 16:46:11,151][52866] Updated weights for policy 1, policy_version 49600 (0.0008) -[2023-10-15 16:46:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 101416960. Throughput: 0: 1776.7, 1: 1804.8. Samples: 25367338. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) -[2023-10-15 16:46:13,442][51532] Avg episode reward: [(0, '46.490'), (1, '45.960')] -[2023-10-15 16:46:14,151][52833] Updated weights for policy 0, policy_version 49450 (0.0008) -[2023-10-15 16:46:14,522][52833] Updated weights for policy 0, policy_version 49460 (0.0009) -[2023-10-15 16:46:14,846][52866] Updated weights for policy 1, policy_version 49610 (0.0008) -[2023-10-15 16:46:14,894][52833] Updated weights for policy 0, policy_version 49470 (0.0009) -[2023-10-15 16:46:15,210][52866] Updated weights for policy 1, policy_version 49620 (0.0008) -[2023-10-15 16:46:15,572][52866] Updated weights for policy 1, policy_version 49630 (0.0008) -[2023-10-15 16:46:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 101482496. Throughput: 0: 1775.0, 1: 1805.6. Samples: 25377282. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) -[2023-10-15 16:46:18,442][51532] Avg episode reward: [(0, '49.540'), (1, '45.060')] -[2023-10-15 16:46:18,501][52833] Updated weights for policy 0, policy_version 49480 (0.0007) -[2023-10-15 16:46:18,871][52833] Updated weights for policy 0, policy_version 49490 (0.0007) -[2023-10-15 16:46:19,248][52833] Updated weights for policy 0, policy_version 49500 (0.0008) -[2023-10-15 16:46:19,251][52866] Updated weights for policy 1, policy_version 49640 (0.0007) -[2023-10-15 16:46:19,616][52866] Updated weights for policy 1, policy_version 49650 (0.0008) -[2023-10-15 16:46:19,975][52866] Updated weights for policy 1, policy_version 49660 (0.0008) -[2023-10-15 16:46:23,162][52833] Updated weights for policy 0, policy_version 49510 (0.0009) -[2023-10-15 16:46:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 101548032. Throughput: 0: 1778.1, 1: 1809.0. Samples: 25399890. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) -[2023-10-15 16:46:23,441][51532] Avg episode reward: [(0, '49.710'), (1, '45.690')] -[2023-10-15 16:46:23,524][52833] Updated weights for policy 0, policy_version 49520 (0.0008) -[2023-10-15 16:46:23,714][52866] Updated weights for policy 1, policy_version 49670 (0.0008) -[2023-10-15 16:46:23,891][52833] Updated weights for policy 0, policy_version 49530 (0.0007) -[2023-10-15 16:46:24,082][52866] Updated weights for policy 1, policy_version 49680 (0.0008) -[2023-10-15 16:46:24,450][52866] Updated weights for policy 1, policy_version 49690 (0.0008) -[2023-10-15 16:46:27,666][52833] Updated weights for policy 0, policy_version 49540 (0.0009) -[2023-10-15 16:46:28,038][52833] Updated weights for policy 0, policy_version 49550 (0.0011) -[2023-10-15 16:46:28,322][52866] Updated weights for policy 1, policy_version 49700 (0.0009) -[2023-10-15 16:46:28,410][52833] Updated weights for policy 0, policy_version 49560 (0.0008) -[2023-10-15 16:46:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 101613568. Throughput: 0: 1798.8, 1: 1809.9. Samples: 25421902. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) -[2023-10-15 16:46:28,441][51532] Avg episode reward: [(0, '48.480'), (1, '46.380')] -[2023-10-15 16:46:28,688][52866] Updated weights for policy 1, policy_version 49710 (0.0007) -[2023-10-15 16:46:29,058][52866] Updated weights for policy 1, policy_version 49720 (0.0009) -[2023-10-15 16:46:32,096][52833] Updated weights for policy 0, policy_version 49570 (0.0007) -[2023-10-15 16:46:32,471][52833] Updated weights for policy 0, policy_version 49580 (0.0007) -[2023-10-15 16:46:32,829][52866] Updated weights for policy 1, policy_version 49730 (0.0007) -[2023-10-15 16:46:32,836][52833] Updated weights for policy 0, policy_version 49590 (0.0008) -[2023-10-15 16:46:33,196][52866] Updated weights for policy 1, policy_version 49740 (0.0007) -[2023-10-15 16:46:33,202][52833] Updated weights for policy 0, policy_version 49600 (0.0007) -[2023-10-15 16:46:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 101711872. Throughput: 0: 1784.2, 1: 1802.5. Samples: 25432122. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) -[2023-10-15 16:46:33,442][51532] Avg episode reward: [(0, '49.790'), (1, '44.550')] -[2023-10-15 16:46:33,568][52866] Updated weights for policy 1, policy_version 49750 (0.0009) -[2023-10-15 16:46:33,933][52866] Updated weights for policy 1, policy_version 49760 (0.0010) -[2023-10-15 16:46:36,836][52833] Updated weights for policy 0, policy_version 49610 (0.0011) -[2023-10-15 16:46:37,207][52833] Updated weights for policy 0, policy_version 49620 (0.0009) -[2023-10-15 16:46:37,567][52833] Updated weights for policy 0, policy_version 49630 (0.0010) -[2023-10-15 16:46:37,706][52866] Updated weights for policy 1, policy_version 49770 (0.0008) -[2023-10-15 16:46:38,083][52866] Updated weights for policy 1, policy_version 49780 (0.0010) -[2023-10-15 16:46:38,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 101777408. Throughput: 0: 1806.6, 1: 1800.6. Samples: 25454222. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) -[2023-10-15 16:46:38,441][51532] Avg episode reward: [(0, '49.230'), (1, '43.350')] -[2023-10-15 16:46:38,449][52866] Updated weights for policy 1, policy_version 49790 (0.0007) -[2023-10-15 16:46:41,416][52833] Updated weights for policy 0, policy_version 49640 (0.0010) -[2023-10-15 16:46:41,794][52833] Updated weights for policy 0, policy_version 49650 (0.0009) -[2023-10-15 16:46:42,154][52833] Updated weights for policy 0, policy_version 49660 (0.0007) -[2023-10-15 16:46:42,271][52866] Updated weights for policy 1, policy_version 49800 (0.0007) -[2023-10-15 16:46:42,641][52866] Updated weights for policy 1, policy_version 49810 (0.0009) -[2023-10-15 16:46:43,007][52866] Updated weights for policy 1, policy_version 49820 (0.0008) -[2023-10-15 16:46:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 101875712. Throughput: 0: 1792.0, 1: 1802.9. Samples: 25474236. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-15 16:46:43,442][51532] Avg episode reward: [(0, '49.150'), (1, '44.920')] -[2023-10-15 16:46:45,896][52833] Updated weights for policy 0, policy_version 49670 (0.0007) -[2023-10-15 16:46:46,266][52833] Updated weights for policy 0, policy_version 49680 (0.0008) -[2023-10-15 16:46:46,639][52833] Updated weights for policy 0, policy_version 49690 (0.0008) -[2023-10-15 16:46:46,771][52866] Updated weights for policy 1, policy_version 49830 (0.0008) -[2023-10-15 16:46:47,136][52866] Updated weights for policy 1, policy_version 49840 (0.0008) -[2023-10-15 16:46:47,496][52866] Updated weights for policy 1, policy_version 49850 (0.0007) -[2023-10-15 16:46:48,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 101941248. Throughput: 0: 1813.2, 1: 1792.8. Samples: 25486632. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-15 16:46:48,442][51532] Avg episode reward: [(0, '51.120'), (1, '43.650')] -[2023-10-15 16:46:50,431][52833] Updated weights for policy 0, policy_version 49700 (0.0008) -[2023-10-15 16:46:50,801][52833] Updated weights for policy 0, policy_version 49710 (0.0008) -[2023-10-15 16:46:51,170][52833] Updated weights for policy 0, policy_version 49720 (0.0009) -[2023-10-15 16:46:51,319][52866] Updated weights for policy 1, policy_version 49860 (0.0008) -[2023-10-15 16:46:51,681][52866] Updated weights for policy 1, policy_version 49870 (0.0008) -[2023-10-15 16:46:52,045][52866] Updated weights for policy 1, policy_version 49880 (0.0010) -[2023-10-15 16:46:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102006784. Throughput: 0: 1801.0, 1: 1801.6. Samples: 25506832. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-15 16:46:53,441][51532] Avg episode reward: [(0, '51.040'), (1, '43.400')] -[2023-10-15 16:46:55,001][52833] Updated weights for policy 0, policy_version 49730 (0.0008) -[2023-10-15 16:46:55,368][52833] Updated weights for policy 0, policy_version 49740 (0.0011) -[2023-10-15 16:46:55,712][52866] Updated weights for policy 1, policy_version 49890 (0.0008) -[2023-10-15 16:46:55,734][52833] Updated weights for policy 0, policy_version 49750 (0.0008) -[2023-10-15 16:46:56,085][52866] Updated weights for policy 1, policy_version 49900 (0.0010) -[2023-10-15 16:46:56,092][52833] Updated weights for policy 0, policy_version 49760 (0.0008) -[2023-10-15 16:46:56,455][52866] Updated weights for policy 1, policy_version 49910 (0.0008) -[2023-10-15 16:46:56,826][52866] Updated weights for policy 1, policy_version 49920 (0.0010) -[2023-10-15 16:46:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102072320. Throughput: 0: 1792.6, 1: 1792.7. Samples: 25528678. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-15 16:46:58,441][51532] Avg episode reward: [(0, '54.980'), (1, '44.800')] -[2023-10-15 16:46:59,861][52833] Updated weights for policy 0, policy_version 49770 (0.0009) -[2023-10-15 16:47:00,227][52833] Updated weights for policy 0, policy_version 49780 (0.0008) -[2023-10-15 16:47:00,492][52866] Updated weights for policy 1, policy_version 49930 (0.0008) -[2023-10-15 16:47:00,592][52833] Updated weights for policy 0, policy_version 49790 (0.0007) -[2023-10-15 16:47:00,856][52866] Updated weights for policy 1, policy_version 49940 (0.0007) -[2023-10-15 16:47:01,220][52866] Updated weights for policy 1, policy_version 49950 (0.0008) -[2023-10-15 16:47:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102137856. Throughput: 0: 1792.7, 1: 1802.7. Samples: 25539074. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-15 16:47:03,442][51532] Avg episode reward: [(0, '53.950'), (1, '44.090')] -[2023-10-15 16:47:04,497][52833] Updated weights for policy 0, policy_version 49800 (0.0009) -[2023-10-15 16:47:04,867][52833] Updated weights for policy 0, policy_version 49810 (0.0008) -[2023-10-15 16:47:04,894][52866] Updated weights for policy 1, policy_version 49960 (0.0007) -[2023-10-15 16:47:05,235][52833] Updated weights for policy 0, policy_version 49820 (0.0007) -[2023-10-15 16:47:05,263][52866] Updated weights for policy 1, policy_version 49970 (0.0008) -[2023-10-15 16:47:05,624][52866] Updated weights for policy 1, policy_version 49980 (0.0008) -[2023-10-15 16:47:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 102203392. Throughput: 0: 1787.5, 1: 1791.9. Samples: 25560964. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-15 16:47:08,441][51532] Avg episode reward: [(0, '52.700'), (1, '44.560')] -[2023-10-15 16:47:09,061][52833] Updated weights for policy 0, policy_version 49830 (0.0009) -[2023-10-15 16:47:09,429][52833] Updated weights for policy 0, policy_version 49840 (0.0007) -[2023-10-15 16:47:09,459][52866] Updated weights for policy 1, policy_version 49990 (0.0007) -[2023-10-15 16:47:09,793][52833] Updated weights for policy 0, policy_version 49850 (0.0010) -[2023-10-15 16:47:09,826][52866] Updated weights for policy 1, policy_version 50000 (0.0009) -[2023-10-15 16:47:10,188][52866] Updated weights for policy 1, policy_version 50010 (0.0010) -[2023-10-15 16:47:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 102268928. Throughput: 0: 1795.2, 1: 1786.3. Samples: 25583072. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-15 16:47:13,442][51532] Avg episode reward: [(0, '52.830'), (1, '44.510')] -[2023-10-15 16:47:13,611][52833] Updated weights for policy 0, policy_version 49860 (0.0007) -[2023-10-15 16:47:13,982][52866] Updated weights for policy 1, policy_version 50020 (0.0008) -[2023-10-15 16:47:14,002][52833] Updated weights for policy 0, policy_version 49870 (0.0008) -[2023-10-15 16:47:14,341][52866] Updated weights for policy 1, policy_version 50030 (0.0008) -[2023-10-15 16:47:14,374][52833] Updated weights for policy 0, policy_version 49880 (0.0009) -[2023-10-15 16:47:14,719][52866] Updated weights for policy 1, policy_version 50040 (0.0009) -[2023-10-15 16:47:18,067][52833] Updated weights for policy 0, policy_version 49890 (0.0009) -[2023-10-15 16:47:18,435][52833] Updated weights for policy 0, policy_version 49900 (0.0007) -[2023-10-15 16:47:18,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 102334464. Throughput: 0: 1782.3, 1: 1786.0. Samples: 25592698. Policy #0 lag: (min: 32.0, avg: 54.8, max: 56.0) -[2023-10-15 16:47:18,442][51532] Avg episode reward: [(0, '52.430'), (1, '45.560')] -[2023-10-15 16:47:18,504][52866] Updated weights for policy 1, policy_version 50050 (0.0009) -[2023-10-15 16:47:18,812][52833] Updated weights for policy 0, policy_version 49910 (0.0008) -[2023-10-15 16:47:18,865][52866] Updated weights for policy 1, policy_version 50060 (0.0009) -[2023-10-15 16:47:19,179][52833] Updated weights for policy 0, policy_version 49920 (0.0008) -[2023-10-15 16:47:19,225][52866] Updated weights for policy 1, policy_version 50070 (0.0009) -[2023-10-15 16:47:19,594][52866] Updated weights for policy 1, policy_version 50080 (0.0008) -[2023-10-15 16:47:23,084][52833] Updated weights for policy 0, policy_version 49930 (0.0008) -[2023-10-15 16:47:23,321][52866] Updated weights for policy 1, policy_version 50090 (0.0007) -[2023-10-15 16:47:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 102400000. Throughput: 0: 1780.5, 1: 1786.2. Samples: 25614726. Policy #0 lag: (min: 32.0, avg: 54.8, max: 56.0) -[2023-10-15 16:47:23,442][51532] Avg episode reward: [(0, '51.510'), (1, '44.100')] -[2023-10-15 16:47:23,451][52833] Updated weights for policy 0, policy_version 49940 (0.0008) -[2023-10-15 16:47:23,676][52866] Updated weights for policy 1, policy_version 50100 (0.0008) -[2023-10-15 16:47:23,817][52833] Updated weights for policy 0, policy_version 49950 (0.0009) -[2023-10-15 16:47:24,040][52866] Updated weights for policy 1, policy_version 50110 (0.0009) -[2023-10-15 16:47:27,444][52833] Updated weights for policy 0, policy_version 49960 (0.0008) -[2023-10-15 16:47:27,774][52866] Updated weights for policy 1, policy_version 50120 (0.0008) -[2023-10-15 16:47:27,807][52833] Updated weights for policy 0, policy_version 49970 (0.0008) -[2023-10-15 16:47:28,136][52866] Updated weights for policy 1, policy_version 50130 (0.0007) -[2023-10-15 16:47:28,173][52833] Updated weights for policy 0, policy_version 49980 (0.0009) -[2023-10-15 16:47:28,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 102498304. Throughput: 0: 1791.3, 1: 1802.9. Samples: 25635970. Policy #0 lag: (min: 32.0, avg: 54.8, max: 56.0) -[2023-10-15 16:47:28,441][51532] Avg episode reward: [(0, '50.940'), (1, '44.410')] -[2023-10-15 16:47:28,503][52866] Updated weights for policy 1, policy_version 50140 (0.0007) -[2023-10-15 16:47:31,865][52833] Updated weights for policy 0, policy_version 49990 (0.0008) -[2023-10-15 16:47:32,238][52833] Updated weights for policy 0, policy_version 50000 (0.0009) -[2023-10-15 16:47:32,343][52866] Updated weights for policy 1, policy_version 50150 (0.0009) -[2023-10-15 16:47:32,608][52833] Updated weights for policy 0, policy_version 50010 (0.0008) -[2023-10-15 16:47:32,710][52866] Updated weights for policy 1, policy_version 50160 (0.0007) -[2023-10-15 16:47:33,069][52866] Updated weights for policy 1, policy_version 50170 (0.0008) -[2023-10-15 16:47:33,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 102596608. Throughput: 0: 1778.3, 1: 1787.8. Samples: 25647104. Policy #0 lag: (min: 32.0, avg: 54.8, max: 56.0) -[2023-10-15 16:47:33,442][51532] Avg episode reward: [(0, '49.000'), (1, '44.520')] -[2023-10-15 16:47:36,344][52833] Updated weights for policy 0, policy_version 50020 (0.0007) -[2023-10-15 16:47:36,699][52833] Updated weights for policy 0, policy_version 50030 (0.0009) -[2023-10-15 16:47:36,700][52866] Updated weights for policy 1, policy_version 50180 (0.0008) -[2023-10-15 16:47:37,056][52866] Updated weights for policy 1, policy_version 50190 (0.0008) -[2023-10-15 16:47:37,073][52833] Updated weights for policy 0, policy_version 50040 (0.0008) -[2023-10-15 16:47:37,427][52866] Updated weights for policy 1, policy_version 50200 (0.0008) -[2023-10-15 16:47:38,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 102662144. Throughput: 0: 1787.9, 1: 1803.7. Samples: 25668454. Policy #0 lag: (min: 32.0, avg: 54.8, max: 56.0) -[2023-10-15 16:47:38,442][51532] Avg episode reward: [(0, '50.730'), (1, '46.120')] -[2023-10-15 16:47:40,819][52833] Updated weights for policy 0, policy_version 50050 (0.0009) -[2023-10-15 16:47:41,190][52833] Updated weights for policy 0, policy_version 50060 (0.0008) -[2023-10-15 16:47:41,299][52866] Updated weights for policy 1, policy_version 50210 (0.0007) -[2023-10-15 16:47:41,548][52833] Updated weights for policy 0, policy_version 50070 (0.0008) -[2023-10-15 16:47:41,663][52866] Updated weights for policy 1, policy_version 50220 (0.0008) -[2023-10-15 16:47:41,921][52833] Updated weights for policy 0, policy_version 50080 (0.0009) -[2023-10-15 16:47:42,027][52866] Updated weights for policy 1, policy_version 50230 (0.0009) -[2023-10-15 16:47:42,393][52866] Updated weights for policy 1, policy_version 50240 (0.0009) -[2023-10-15 16:47:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102727680. Throughput: 0: 1773.5, 1: 1790.2. Samples: 25689046. Policy #0 lag: (min: 32.0, avg: 54.8, max: 56.0) -[2023-10-15 16:47:43,442][51532] Avg episode reward: [(0, '53.890'), (1, '44.680')] -[2023-10-15 16:47:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000050080_51281920.pth... -[2023-10-15 16:47:43,453][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000050240_51445760.pth... -[2023-10-15 16:47:43,481][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000048416_49577984.pth -[2023-10-15 16:47:43,485][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000050080_51281920.pth -[2023-10-15 16:47:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000048544_49709056.pth -[2023-10-15 16:47:43,492][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000050240_51445760.pth -[2023-10-15 16:47:45,847][52833] Updated weights for policy 0, policy_version 50090 (0.0007) -[2023-10-15 16:47:46,045][52866] Updated weights for policy 1, policy_version 50250 (0.0008) -[2023-10-15 16:47:46,213][52833] Updated weights for policy 0, policy_version 50100 (0.0009) -[2023-10-15 16:47:46,403][52866] Updated weights for policy 1, policy_version 50260 (0.0008) -[2023-10-15 16:47:46,588][52833] Updated weights for policy 0, policy_version 50110 (0.0009) -[2023-10-15 16:47:46,765][52866] Updated weights for policy 1, policy_version 50270 (0.0009) -[2023-10-15 16:47:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102793216. Throughput: 0: 1793.6, 1: 1800.3. Samples: 25700796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:47:48,442][51532] Avg episode reward: [(0, '52.160'), (1, '40.770')] -[2023-10-15 16:47:50,463][52833] Updated weights for policy 0, policy_version 50120 (0.0007) -[2023-10-15 16:47:50,585][52866] Updated weights for policy 1, policy_version 50280 (0.0008) -[2023-10-15 16:47:50,819][52833] Updated weights for policy 0, policy_version 50130 (0.0009) -[2023-10-15 16:47:50,951][52866] Updated weights for policy 1, policy_version 50290 (0.0008) -[2023-10-15 16:47:51,195][52833] Updated weights for policy 0, policy_version 50140 (0.0009) -[2023-10-15 16:47:51,319][52866] Updated weights for policy 1, policy_version 50300 (0.0008) -[2023-10-15 16:47:53,441][51532] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102858752. Throughput: 0: 1770.2, 1: 1783.4. Samples: 25720878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:47:53,442][51532] Avg episode reward: [(0, '52.660'), (1, '38.250')] -[2023-10-15 16:47:54,757][52833] Updated weights for policy 0, policy_version 50150 (0.0008) -[2023-10-15 16:47:55,011][52866] Updated weights for policy 1, policy_version 50310 (0.0009) -[2023-10-15 16:47:55,115][52833] Updated weights for policy 0, policy_version 50160 (0.0009) -[2023-10-15 16:47:55,378][52866] Updated weights for policy 1, policy_version 50320 (0.0008) -[2023-10-15 16:47:55,481][52833] Updated weights for policy 0, policy_version 50170 (0.0007) -[2023-10-15 16:47:55,745][52866] Updated weights for policy 1, policy_version 50330 (0.0009) -[2023-10-15 16:47:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 102924288. Throughput: 0: 1776.6, 1: 1787.3. Samples: 25743450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:47:58,442][51532] Avg episode reward: [(0, '52.830'), (1, '40.460')] -[2023-10-15 16:47:59,225][52833] Updated weights for policy 0, policy_version 50180 (0.0008) -[2023-10-15 16:47:59,615][52833] Updated weights for policy 0, policy_version 50190 (0.0008) -[2023-10-15 16:47:59,620][52866] Updated weights for policy 1, policy_version 50340 (0.0008) -[2023-10-15 16:47:59,979][52833] Updated weights for policy 0, policy_version 50200 (0.0010) -[2023-10-15 16:47:59,980][52866] Updated weights for policy 1, policy_version 50350 (0.0008) -[2023-10-15 16:48:00,346][52866] Updated weights for policy 1, policy_version 50360 (0.0008) -[2023-10-15 16:48:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 102989824. Throughput: 0: 1780.4, 1: 1788.1. Samples: 25753280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:48:03,441][51532] Avg episode reward: [(0, '52.510'), (1, '41.300')] -[2023-10-15 16:48:03,796][52833] Updated weights for policy 0, policy_version 50210 (0.0009) -[2023-10-15 16:48:04,126][52866] Updated weights for policy 1, policy_version 50370 (0.0009) -[2023-10-15 16:48:04,161][52833] Updated weights for policy 0, policy_version 50220 (0.0009) -[2023-10-15 16:48:04,487][52866] Updated weights for policy 1, policy_version 50380 (0.0009) -[2023-10-15 16:48:04,528][52833] Updated weights for policy 0, policy_version 50230 (0.0007) -[2023-10-15 16:48:04,843][52866] Updated weights for policy 1, policy_version 50390 (0.0007) -[2023-10-15 16:48:04,894][52833] Updated weights for policy 0, policy_version 50240 (0.0007) -[2023-10-15 16:48:05,206][52866] Updated weights for policy 1, policy_version 50400 (0.0007) -[2023-10-15 16:48:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 103055360. Throughput: 0: 1784.9, 1: 1793.5. Samples: 25775752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:48:08,442][51532] Avg episode reward: [(0, '50.420'), (1, '43.070')] -[2023-10-15 16:48:08,670][52833] Updated weights for policy 0, policy_version 50250 (0.0007) -[2023-10-15 16:48:08,917][52866] Updated weights for policy 1, policy_version 50410 (0.0008) -[2023-10-15 16:48:09,039][52833] Updated weights for policy 0, policy_version 50260 (0.0007) -[2023-10-15 16:48:09,280][52866] Updated weights for policy 1, policy_version 50420 (0.0008) -[2023-10-15 16:48:09,410][52833] Updated weights for policy 0, policy_version 50270 (0.0007) -[2023-10-15 16:48:09,636][52866] Updated weights for policy 1, policy_version 50430 (0.0008) -[2023-10-15 16:48:13,203][52833] Updated weights for policy 0, policy_version 50280 (0.0009) -[2023-10-15 16:48:13,375][52866] Updated weights for policy 1, policy_version 50440 (0.0008) -[2023-10-15 16:48:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 103120896. Throughput: 0: 1798.0, 1: 1808.9. Samples: 25798282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:48:13,442][51532] Avg episode reward: [(0, '49.610'), (1, '40.190')] -[2023-10-15 16:48:13,577][52833] Updated weights for policy 0, policy_version 50290 (0.0009) -[2023-10-15 16:48:13,738][52866] Updated weights for policy 1, policy_version 50450 (0.0007) -[2023-10-15 16:48:13,934][52833] Updated weights for policy 0, policy_version 50300 (0.0011) -[2023-10-15 16:48:14,102][52866] Updated weights for policy 1, policy_version 50460 (0.0008) -[2023-10-15 16:48:17,753][52833] Updated weights for policy 0, policy_version 50310 (0.0008) -[2023-10-15 16:48:17,938][52866] Updated weights for policy 1, policy_version 50470 (0.0008) -[2023-10-15 16:48:18,127][52833] Updated weights for policy 0, policy_version 50320 (0.0008) -[2023-10-15 16:48:18,316][52866] Updated weights for policy 1, policy_version 50480 (0.0007) -[2023-10-15 16:48:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 103186432. Throughput: 0: 1780.3, 1: 1796.3. Samples: 25808052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:48:18,441][51532] Avg episode reward: [(0, '51.140'), (1, '40.280')] -[2023-10-15 16:48:18,499][52833] Updated weights for policy 0, policy_version 50330 (0.0009) -[2023-10-15 16:48:18,675][52866] Updated weights for policy 1, policy_version 50490 (0.0007) -[2023-10-15 16:48:22,231][52833] Updated weights for policy 0, policy_version 50340 (0.0008) -[2023-10-15 16:48:22,399][52866] Updated weights for policy 1, policy_version 50500 (0.0008) -[2023-10-15 16:48:22,604][52833] Updated weights for policy 0, policy_version 50350 (0.0008) -[2023-10-15 16:48:22,771][52866] Updated weights for policy 1, policy_version 50510 (0.0009) -[2023-10-15 16:48:22,974][52833] Updated weights for policy 0, policy_version 50360 (0.0010) -[2023-10-15 16:48:23,134][52866] Updated weights for policy 1, policy_version 50520 (0.0008) -[2023-10-15 16:48:23,441][51532] Fps is (10 sec: 19661.4, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 103317504. Throughput: 0: 1797.4, 1: 1802.8. Samples: 25830462. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) -[2023-10-15 16:48:23,441][51532] Avg episode reward: [(0, '51.850'), (1, '39.760')] -[2023-10-15 16:48:26,676][52833] Updated weights for policy 0, policy_version 50370 (0.0008) -[2023-10-15 16:48:26,859][52866] Updated weights for policy 1, policy_version 50530 (0.0009) -[2023-10-15 16:48:27,045][52833] Updated weights for policy 0, policy_version 50380 (0.0009) -[2023-10-15 16:48:27,258][52866] Updated weights for policy 1, policy_version 50540 (0.0009) -[2023-10-15 16:48:27,421][52833] Updated weights for policy 0, policy_version 50390 (0.0008) -[2023-10-15 16:48:27,622][52866] Updated weights for policy 1, policy_version 50550 (0.0009) -[2023-10-15 16:48:27,797][52833] Updated weights for policy 0, policy_version 50400 (0.0008) -[2023-10-15 16:48:27,988][52866] Updated weights for policy 1, policy_version 50560 (0.0010) -[2023-10-15 16:48:28,441][51532] Fps is (10 sec: 19660.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 103383040. Throughput: 0: 1781.4, 1: 1794.7. Samples: 25849970. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) -[2023-10-15 16:48:28,442][51532] Avg episode reward: [(0, '48.920'), (1, '42.920')] -[2023-10-15 16:48:31,671][52866] Updated weights for policy 1, policy_version 50570 (0.0007) -[2023-10-15 16:48:31,678][52833] Updated weights for policy 0, policy_version 50410 (0.0007) -[2023-10-15 16:48:32,030][52866] Updated weights for policy 1, policy_version 50580 (0.0007) -[2023-10-15 16:48:32,051][52833] Updated weights for policy 0, policy_version 50420 (0.0007) -[2023-10-15 16:48:32,402][52866] Updated weights for policy 1, policy_version 50590 (0.0007) -[2023-10-15 16:48:32,425][52833] Updated weights for policy 0, policy_version 50430 (0.0007) -[2023-10-15 16:48:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103448576. Throughput: 0: 1796.7, 1: 1803.6. Samples: 25862808. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) -[2023-10-15 16:48:33,442][51532] Avg episode reward: [(0, '48.820'), (1, '44.310')] -[2023-10-15 16:48:36,140][52833] Updated weights for policy 0, policy_version 50440 (0.0007) -[2023-10-15 16:48:36,156][52866] Updated weights for policy 1, policy_version 50600 (0.0008) -[2023-10-15 16:48:36,511][52833] Updated weights for policy 0, policy_version 50450 (0.0009) -[2023-10-15 16:48:36,514][52866] Updated weights for policy 1, policy_version 50610 (0.0007) -[2023-10-15 16:48:36,872][52833] Updated weights for policy 0, policy_version 50460 (0.0009) -[2023-10-15 16:48:36,886][52866] Updated weights for policy 1, policy_version 50620 (0.0008) -[2023-10-15 16:48:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103514112. Throughput: 0: 1795.2, 1: 1798.5. Samples: 25882598. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) -[2023-10-15 16:48:38,441][51532] Avg episode reward: [(0, '49.860'), (1, '45.490')] -[2023-10-15 16:48:40,509][52866] Updated weights for policy 1, policy_version 50630 (0.0009) -[2023-10-15 16:48:40,707][52833] Updated weights for policy 0, policy_version 50470 (0.0008) -[2023-10-15 16:48:40,880][52866] Updated weights for policy 1, policy_version 50640 (0.0009) -[2023-10-15 16:48:41,073][52833] Updated weights for policy 0, policy_version 50480 (0.0008) -[2023-10-15 16:48:41,247][52866] Updated weights for policy 1, policy_version 50650 (0.0008) -[2023-10-15 16:48:41,447][52833] Updated weights for policy 0, policy_version 50490 (0.0008) -[2023-10-15 16:48:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 103579648. Throughput: 0: 1781.4, 1: 1800.1. Samples: 25904618. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) -[2023-10-15 16:48:43,442][51532] Avg episode reward: [(0, '49.050'), (1, '46.810')] -[2023-10-15 16:48:45,138][52866] Updated weights for policy 1, policy_version 50660 (0.0008) -[2023-10-15 16:48:45,370][52833] Updated weights for policy 0, policy_version 50500 (0.0008) -[2023-10-15 16:48:45,515][52866] Updated weights for policy 1, policy_version 50670 (0.0008) -[2023-10-15 16:48:45,767][52833] Updated weights for policy 0, policy_version 50510 (0.0007) -[2023-10-15 16:48:45,878][52866] Updated weights for policy 1, policy_version 50680 (0.0007) -[2023-10-15 16:48:46,136][52833] Updated weights for policy 0, policy_version 50520 (0.0008) -[2023-10-15 16:48:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 103645184. Throughput: 0: 1791.2, 1: 1801.7. Samples: 25914962. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) -[2023-10-15 16:48:48,442][51532] Avg episode reward: [(0, '49.970'), (1, '46.860')] -[2023-10-15 16:48:49,757][52866] Updated weights for policy 1, policy_version 50690 (0.0008) -[2023-10-15 16:48:49,804][52833] Updated weights for policy 0, policy_version 50530 (0.0008) -[2023-10-15 16:48:50,122][52866] Updated weights for policy 1, policy_version 50700 (0.0007) -[2023-10-15 16:48:50,167][52833] Updated weights for policy 0, policy_version 50540 (0.0007) -[2023-10-15 16:48:50,491][52866] Updated weights for policy 1, policy_version 50710 (0.0007) -[2023-10-15 16:48:50,539][52833] Updated weights for policy 0, policy_version 50550 (0.0008) -[2023-10-15 16:48:50,858][52866] Updated weights for policy 1, policy_version 50720 (0.0007) -[2023-10-15 16:48:50,907][52833] Updated weights for policy 0, policy_version 50560 (0.0008) -[2023-10-15 16:48:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 103710720. Throughput: 0: 1776.4, 1: 1788.8. Samples: 25936186. Policy #0 lag: (min: 14.0, avg: 23.2, max: 46.0) -[2023-10-15 16:48:53,442][51532] Avg episode reward: [(0, '53.410'), (1, '48.290')] -[2023-10-15 16:48:54,601][52866] Updated weights for policy 1, policy_version 50730 (0.0008) -[2023-10-15 16:48:54,672][52833] Updated weights for policy 0, policy_version 50570 (0.0008) -[2023-10-15 16:48:54,963][52866] Updated weights for policy 1, policy_version 50740 (0.0010) -[2023-10-15 16:48:55,034][52833] Updated weights for policy 0, policy_version 50580 (0.0007) -[2023-10-15 16:48:55,320][52866] Updated weights for policy 1, policy_version 50750 (0.0007) -[2023-10-15 16:48:55,404][52833] Updated weights for policy 0, policy_version 50590 (0.0008) -[2023-10-15 16:48:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 103776256. Throughput: 0: 1778.7, 1: 1791.7. Samples: 25958946. Policy #0 lag: (min: 29.0, avg: 30.9, max: 58.0) -[2023-10-15 16:48:58,442][51532] Avg episode reward: [(0, '54.820'), (1, '45.930')] -[2023-10-15 16:48:59,094][52833] Updated weights for policy 0, policy_version 50600 (0.0007) -[2023-10-15 16:48:59,117][52866] Updated weights for policy 1, policy_version 50760 (0.0008) -[2023-10-15 16:48:59,457][52833] Updated weights for policy 0, policy_version 50610 (0.0008) -[2023-10-15 16:48:59,475][52866] Updated weights for policy 1, policy_version 50770 (0.0007) -[2023-10-15 16:48:59,827][52833] Updated weights for policy 0, policy_version 50620 (0.0008) -[2023-10-15 16:48:59,846][52866] Updated weights for policy 1, policy_version 50780 (0.0007) -[2023-10-15 16:49:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 103841792. Throughput: 0: 1775.9, 1: 1793.5. Samples: 25968678. Policy #0 lag: (min: 29.0, avg: 30.9, max: 58.0) -[2023-10-15 16:49:03,442][51532] Avg episode reward: [(0, '54.400'), (1, '45.760')] -[2023-10-15 16:49:03,497][52866] Updated weights for policy 1, policy_version 50790 (0.0007) -[2023-10-15 16:49:03,671][52833] Updated weights for policy 0, policy_version 50630 (0.0009) -[2023-10-15 16:49:03,863][52866] Updated weights for policy 1, policy_version 50800 (0.0007) -[2023-10-15 16:49:04,040][52833] Updated weights for policy 0, policy_version 50640 (0.0009) -[2023-10-15 16:49:04,235][52866] Updated weights for policy 1, policy_version 50810 (0.0007) -[2023-10-15 16:49:04,403][52833] Updated weights for policy 0, policy_version 50650 (0.0009) -[2023-10-15 16:49:07,891][52866] Updated weights for policy 1, policy_version 50820 (0.0007) -[2023-10-15 16:49:08,137][52833] Updated weights for policy 0, policy_version 50660 (0.0010) -[2023-10-15 16:49:08,262][52866] Updated weights for policy 1, policy_version 50830 (0.0009) -[2023-10-15 16:49:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 103907328. Throughput: 0: 1776.8, 1: 1797.1. Samples: 25991292. Policy #0 lag: (min: 29.0, avg: 30.9, max: 58.0) -[2023-10-15 16:49:08,442][51532] Avg episode reward: [(0, '52.680'), (1, '45.640')] -[2023-10-15 16:49:08,507][52833] Updated weights for policy 0, policy_version 50670 (0.0008) -[2023-10-15 16:49:08,628][52866] Updated weights for policy 1, policy_version 50840 (0.0009) -[2023-10-15 16:49:08,878][52833] Updated weights for policy 0, policy_version 50680 (0.0009) -[2023-10-15 16:49:12,258][52866] Updated weights for policy 1, policy_version 50850 (0.0008) -[2023-10-15 16:49:12,630][52866] Updated weights for policy 1, policy_version 50860 (0.0007) -[2023-10-15 16:49:12,658][52833] Updated weights for policy 0, policy_version 50690 (0.0010) -[2023-10-15 16:49:12,996][52866] Updated weights for policy 1, policy_version 50870 (0.0009) -[2023-10-15 16:49:13,029][52833] Updated weights for policy 0, policy_version 50700 (0.0007) -[2023-10-15 16:49:13,356][52866] Updated weights for policy 1, policy_version 50880 (0.0007) -[2023-10-15 16:49:13,395][52833] Updated weights for policy 0, policy_version 50710 (0.0007) -[2023-10-15 16:49:13,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 104005632. Throughput: 0: 1803.3, 1: 1804.3. Samples: 26012314. Policy #0 lag: (min: 29.0, avg: 30.9, max: 58.0) -[2023-10-15 16:49:13,441][51532] Avg episode reward: [(0, '52.540'), (1, '41.990')] -[2023-10-15 16:49:13,769][52833] Updated weights for policy 0, policy_version 50720 (0.0008) -[2023-10-15 16:49:17,210][52866] Updated weights for policy 1, policy_version 50890 (0.0009) -[2023-10-15 16:49:17,580][52866] Updated weights for policy 1, policy_version 50900 (0.0007) -[2023-10-15 16:49:17,591][52833] Updated weights for policy 0, policy_version 50730 (0.0007) -[2023-10-15 16:49:17,946][52833] Updated weights for policy 0, policy_version 50740 (0.0008) -[2023-10-15 16:49:17,950][52866] Updated weights for policy 1, policy_version 50910 (0.0008) -[2023-10-15 16:49:18,314][52833] Updated weights for policy 0, policy_version 50750 (0.0008) -[2023-10-15 16:49:18,441][51532] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 104103936. Throughput: 0: 1772.4, 1: 1790.3. Samples: 26023128. Policy #0 lag: (min: 29.0, avg: 30.9, max: 58.0) -[2023-10-15 16:49:18,441][51532] Avg episode reward: [(0, '54.410'), (1, '41.970')] -[2023-10-15 16:49:21,645][52866] Updated weights for policy 1, policy_version 50920 (0.0009) -[2023-10-15 16:49:22,013][52866] Updated weights for policy 1, policy_version 50930 (0.0010) -[2023-10-15 16:49:22,282][52833] Updated weights for policy 0, policy_version 50760 (0.0008) -[2023-10-15 16:49:22,377][52866] Updated weights for policy 1, policy_version 50940 (0.0008) -[2023-10-15 16:49:22,659][52833] Updated weights for policy 0, policy_version 50770 (0.0009) -[2023-10-15 16:49:23,032][52833] Updated weights for policy 0, policy_version 50780 (0.0011) -[2023-10-15 16:49:23,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 104169472. Throughput: 0: 1797.2, 1: 1806.0. Samples: 26044740. Policy #0 lag: (min: 29.0, avg: 30.9, max: 58.0) -[2023-10-15 16:49:23,441][51532] Avg episode reward: [(0, '49.320'), (1, '42.290')] -[2023-10-15 16:49:26,234][52866] Updated weights for policy 1, policy_version 50950 (0.0009) -[2023-10-15 16:49:26,593][52866] Updated weights for policy 1, policy_version 50960 (0.0010) -[2023-10-15 16:49:26,747][52833] Updated weights for policy 0, policy_version 50790 (0.0009) -[2023-10-15 16:49:26,964][52866] Updated weights for policy 1, policy_version 50970 (0.0007) -[2023-10-15 16:49:27,111][52833] Updated weights for policy 0, policy_version 50800 (0.0007) -[2023-10-15 16:49:27,475][52833] Updated weights for policy 0, policy_version 50810 (0.0010) -[2023-10-15 16:49:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104235008. Throughput: 0: 1773.0, 1: 1787.1. Samples: 26064826. Policy #0 lag: (min: 29.0, avg: 30.9, max: 58.0) -[2023-10-15 16:49:28,442][51532] Avg episode reward: [(0, '49.770'), (1, '41.350')] -[2023-10-15 16:49:30,684][52866] Updated weights for policy 1, policy_version 50980 (0.0009) -[2023-10-15 16:49:31,055][52866] Updated weights for policy 1, policy_version 50990 (0.0008) -[2023-10-15 16:49:31,252][52833] Updated weights for policy 0, policy_version 50820 (0.0007) -[2023-10-15 16:49:31,408][52866] Updated weights for policy 1, policy_version 51000 (0.0009) -[2023-10-15 16:49:31,634][52833] Updated weights for policy 0, policy_version 50830 (0.0009) -[2023-10-15 16:49:31,998][52833] Updated weights for policy 0, policy_version 50840 (0.0010) -[2023-10-15 16:49:33,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104300544. Throughput: 0: 1797.1, 1: 1805.4. Samples: 26077072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:49:33,442][51532] Avg episode reward: [(0, '49.460'), (1, '42.560')] -[2023-10-15 16:49:35,219][52866] Updated weights for policy 1, policy_version 51010 (0.0008) -[2023-10-15 16:49:35,584][52866] Updated weights for policy 1, policy_version 51020 (0.0009) -[2023-10-15 16:49:35,699][52833] Updated weights for policy 0, policy_version 50850 (0.0010) -[2023-10-15 16:49:35,947][52866] Updated weights for policy 1, policy_version 51030 (0.0007) -[2023-10-15 16:49:36,069][52833] Updated weights for policy 0, policy_version 50860 (0.0008) -[2023-10-15 16:49:36,311][52866] Updated weights for policy 1, policy_version 51040 (0.0009) -[2023-10-15 16:49:36,434][52833] Updated weights for policy 0, policy_version 50870 (0.0009) -[2023-10-15 16:49:36,794][52833] Updated weights for policy 0, policy_version 50880 (0.0009) -[2023-10-15 16:49:38,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 104366080. Throughput: 0: 1782.1, 1: 1796.2. Samples: 26097210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:49:38,441][51532] Avg episode reward: [(0, '48.540'), (1, '41.650')] -[2023-10-15 16:49:40,126][52866] Updated weights for policy 1, policy_version 51050 (0.0008) -[2023-10-15 16:49:40,490][52866] Updated weights for policy 1, policy_version 51060 (0.0007) -[2023-10-15 16:49:40,554][52833] Updated weights for policy 0, policy_version 50890 (0.0010) -[2023-10-15 16:49:40,848][52866] Updated weights for policy 1, policy_version 51070 (0.0008) -[2023-10-15 16:49:40,923][52833] Updated weights for policy 0, policy_version 50900 (0.0007) -[2023-10-15 16:49:41,291][52833] Updated weights for policy 0, policy_version 50910 (0.0008) -[2023-10-15 16:49:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 104431616. Throughput: 0: 1778.3, 1: 1791.5. Samples: 26119590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:49:43,442][51532] Avg episode reward: [(0, '48.020'), (1, '42.700')] -[2023-10-15 16:49:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000050912_52133888.pth... -[2023-10-15 16:49:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000051072_52297728.pth... -[2023-10-15 16:49:43,486][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000049408_50593792.pth -[2023-10-15 16:49:43,494][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000049248_50429952.pth -[2023-10-15 16:49:44,723][52866] Updated weights for policy 1, policy_version 51080 (0.0008) -[2023-10-15 16:49:45,090][52866] Updated weights for policy 1, policy_version 51090 (0.0009) -[2023-10-15 16:49:45,185][52833] Updated weights for policy 0, policy_version 50920 (0.0008) -[2023-10-15 16:49:45,453][52866] Updated weights for policy 1, policy_version 51100 (0.0009) -[2023-10-15 16:49:45,551][52833] Updated weights for policy 0, policy_version 50930 (0.0007) -[2023-10-15 16:49:45,918][52833] Updated weights for policy 0, policy_version 50940 (0.0008) -[2023-10-15 16:49:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 104497152. Throughput: 0: 1784.3, 1: 1789.8. Samples: 26129512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:49:48,441][51532] Avg episode reward: [(0, '49.710'), (1, '42.760')] -[2023-10-15 16:49:49,315][52866] Updated weights for policy 1, policy_version 51110 (0.0007) -[2023-10-15 16:49:49,623][52833] Updated weights for policy 0, policy_version 50950 (0.0009) -[2023-10-15 16:49:49,682][52866] Updated weights for policy 1, policy_version 51120 (0.0008) -[2023-10-15 16:49:49,996][52833] Updated weights for policy 0, policy_version 50960 (0.0011) -[2023-10-15 16:49:50,036][52866] Updated weights for policy 1, policy_version 51130 (0.0007) -[2023-10-15 16:49:50,371][52833] Updated weights for policy 0, policy_version 50970 (0.0009) -[2023-10-15 16:49:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 104562688. Throughput: 0: 1775.7, 1: 1781.2. Samples: 26151350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:49:53,441][51532] Avg episode reward: [(0, '48.960'), (1, '43.680')] -[2023-10-15 16:49:53,907][52866] Updated weights for policy 1, policy_version 51140 (0.0007) -[2023-10-15 16:49:54,023][52833] Updated weights for policy 0, policy_version 50980 (0.0007) -[2023-10-15 16:49:54,280][52866] Updated weights for policy 1, policy_version 51150 (0.0007) -[2023-10-15 16:49:54,394][52833] Updated weights for policy 0, policy_version 50990 (0.0007) -[2023-10-15 16:49:54,649][52866] Updated weights for policy 1, policy_version 51160 (0.0007) -[2023-10-15 16:49:54,759][52833] Updated weights for policy 0, policy_version 51000 (0.0008) -[2023-10-15 16:49:58,378][52866] Updated weights for policy 1, policy_version 51170 (0.0009) -[2023-10-15 16:49:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 104628224. Throughput: 0: 1786.8, 1: 1804.4. Samples: 26173918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:49:58,441][51532] Avg episode reward: [(0, '47.100'), (1, '43.490')] -[2023-10-15 16:49:58,500][52833] Updated weights for policy 0, policy_version 51010 (0.0009) -[2023-10-15 16:49:58,786][52866] Updated weights for policy 1, policy_version 51180 (0.0008) -[2023-10-15 16:49:58,875][52833] Updated weights for policy 0, policy_version 51020 (0.0009) -[2023-10-15 16:49:59,149][52866] Updated weights for policy 1, policy_version 51190 (0.0007) -[2023-10-15 16:49:59,238][52833] Updated weights for policy 0, policy_version 51030 (0.0008) -[2023-10-15 16:49:59,517][52866] Updated weights for policy 1, policy_version 51200 (0.0008) -[2023-10-15 16:49:59,599][52833] Updated weights for policy 0, policy_version 51040 (0.0008) -[2023-10-15 16:50:03,286][52833] Updated weights for policy 0, policy_version 51050 (0.0008) -[2023-10-15 16:50:03,362][52866] Updated weights for policy 1, policy_version 51210 (0.0010) -[2023-10-15 16:50:03,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 104693760. Throughput: 0: 1785.8, 1: 1779.6. Samples: 26183570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:03,442][51532] Avg episode reward: [(0, '45.800'), (1, '42.680')] -[2023-10-15 16:50:03,660][52833] Updated weights for policy 0, policy_version 51060 (0.0008) -[2023-10-15 16:50:03,723][52866] Updated weights for policy 1, policy_version 51220 (0.0011) -[2023-10-15 16:50:04,026][52833] Updated weights for policy 0, policy_version 51070 (0.0008) -[2023-10-15 16:50:04,086][52866] Updated weights for policy 1, policy_version 51230 (0.0009) -[2023-10-15 16:50:07,784][52833] Updated weights for policy 0, policy_version 51080 (0.0007) -[2023-10-15 16:50:07,877][52866] Updated weights for policy 1, policy_version 51240 (0.0009) -[2023-10-15 16:50:08,158][52833] Updated weights for policy 0, policy_version 51090 (0.0007) -[2023-10-15 16:50:08,247][52866] Updated weights for policy 1, policy_version 51250 (0.0007) -[2023-10-15 16:50:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 104759296. Throughput: 0: 1790.4, 1: 1790.4. Samples: 26205876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:08,441][51532] Avg episode reward: [(0, '48.600'), (1, '43.730')] -[2023-10-15 16:50:08,518][52833] Updated weights for policy 0, policy_version 51100 (0.0008) -[2023-10-15 16:50:08,609][52866] Updated weights for policy 1, policy_version 51260 (0.0008) -[2023-10-15 16:50:12,115][52866] Updated weights for policy 1, policy_version 51270 (0.0007) -[2023-10-15 16:50:12,315][52833] Updated weights for policy 0, policy_version 51110 (0.0007) -[2023-10-15 16:50:12,483][52866] Updated weights for policy 1, policy_version 51280 (0.0007) -[2023-10-15 16:50:12,680][52833] Updated weights for policy 0, policy_version 51120 (0.0007) -[2023-10-15 16:50:12,847][52866] Updated weights for policy 1, policy_version 51290 (0.0007) -[2023-10-15 16:50:13,042][52833] Updated weights for policy 0, policy_version 51130 (0.0007) -[2023-10-15 16:50:13,441][51532] Fps is (10 sec: 19661.4, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 104890368. Throughput: 0: 1804.1, 1: 1785.6. Samples: 26226360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:13,441][51532] Avg episode reward: [(0, '51.160'), (1, '45.040')] -[2023-10-15 16:50:16,613][52866] Updated weights for policy 1, policy_version 51300 (0.0008) -[2023-10-15 16:50:16,972][52866] Updated weights for policy 1, policy_version 51310 (0.0007) -[2023-10-15 16:50:17,009][52833] Updated weights for policy 0, policy_version 51140 (0.0009) -[2023-10-15 16:50:17,333][52866] Updated weights for policy 1, policy_version 51320 (0.0007) -[2023-10-15 16:50:17,395][52833] Updated weights for policy 0, policy_version 51150 (0.0009) -[2023-10-15 16:50:17,758][52833] Updated weights for policy 0, policy_version 51160 (0.0007) -[2023-10-15 16:50:18,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104955904. Throughput: 0: 1788.7, 1: 1794.0. Samples: 26238294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:18,442][51532] Avg episode reward: [(0, '49.360'), (1, '46.260')] -[2023-10-15 16:50:20,995][52866] Updated weights for policy 1, policy_version 51330 (0.0008) -[2023-10-15 16:50:21,357][52866] Updated weights for policy 1, policy_version 51340 (0.0007) -[2023-10-15 16:50:21,581][52833] Updated weights for policy 0, policy_version 51170 (0.0008) -[2023-10-15 16:50:21,721][52866] Updated weights for policy 1, policy_version 51350 (0.0010) -[2023-10-15 16:50:21,943][52833] Updated weights for policy 0, policy_version 51180 (0.0007) -[2023-10-15 16:50:22,085][52866] Updated weights for policy 1, policy_version 51360 (0.0009) -[2023-10-15 16:50:22,307][52833] Updated weights for policy 0, policy_version 51190 (0.0010) -[2023-10-15 16:50:22,677][52833] Updated weights for policy 0, policy_version 51200 (0.0011) -[2023-10-15 16:50:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105021440. Throughput: 0: 1802.1, 1: 1794.0. Samples: 26259034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:23,441][51532] Avg episode reward: [(0, '49.490'), (1, '46.660')] -[2023-10-15 16:50:25,896][52866] Updated weights for policy 1, policy_version 51370 (0.0009) -[2023-10-15 16:50:26,276][52866] Updated weights for policy 1, policy_version 51380 (0.0009) -[2023-10-15 16:50:26,300][52833] Updated weights for policy 0, policy_version 51210 (0.0007) -[2023-10-15 16:50:26,640][52866] Updated weights for policy 1, policy_version 51390 (0.0007) -[2023-10-15 16:50:26,676][52833] Updated weights for policy 0, policy_version 51220 (0.0007) -[2023-10-15 16:50:27,040][52833] Updated weights for policy 0, policy_version 51230 (0.0009) -[2023-10-15 16:50:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105086976. Throughput: 0: 1784.1, 1: 1786.5. Samples: 26280268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:28,441][51532] Avg episode reward: [(0, '50.520'), (1, '44.660')] -[2023-10-15 16:50:30,479][52866] Updated weights for policy 1, policy_version 51400 (0.0008) -[2023-10-15 16:50:30,846][52833] Updated weights for policy 0, policy_version 51240 (0.0008) -[2023-10-15 16:50:30,851][52866] Updated weights for policy 1, policy_version 51410 (0.0007) -[2023-10-15 16:50:31,215][52833] Updated weights for policy 0, policy_version 51250 (0.0007) -[2023-10-15 16:50:31,224][52866] Updated weights for policy 1, policy_version 51420 (0.0008) -[2023-10-15 16:50:31,592][52833] Updated weights for policy 0, policy_version 51260 (0.0008) -[2023-10-15 16:50:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 105152512. Throughput: 0: 1801.0, 1: 1798.3. Samples: 26291482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:33,442][51532] Avg episode reward: [(0, '49.030'), (1, '43.980')] -[2023-10-15 16:50:34,897][52866] Updated weights for policy 1, policy_version 51430 (0.0009) -[2023-10-15 16:50:35,267][52866] Updated weights for policy 1, policy_version 51440 (0.0007) -[2023-10-15 16:50:35,455][52833] Updated weights for policy 0, policy_version 51270 (0.0008) -[2023-10-15 16:50:35,636][52866] Updated weights for policy 1, policy_version 51450 (0.0007) -[2023-10-15 16:50:35,833][52833] Updated weights for policy 0, policy_version 51280 (0.0008) -[2023-10-15 16:50:36,195][52833] Updated weights for policy 0, policy_version 51290 (0.0009) -[2023-10-15 16:50:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 105218048. Throughput: 0: 1779.4, 1: 1792.8. Samples: 26312098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:38,441][51532] Avg episode reward: [(0, '50.760'), (1, '43.100')] -[2023-10-15 16:50:39,338][52866] Updated weights for policy 1, policy_version 51460 (0.0009) -[2023-10-15 16:50:39,703][52866] Updated weights for policy 1, policy_version 51470 (0.0007) -[2023-10-15 16:50:39,967][52833] Updated weights for policy 0, policy_version 51300 (0.0009) -[2023-10-15 16:50:40,063][52866] Updated weights for policy 1, policy_version 51480 (0.0008) -[2023-10-15 16:50:40,329][52833] Updated weights for policy 0, policy_version 51310 (0.0008) -[2023-10-15 16:50:40,699][52833] Updated weights for policy 0, policy_version 51320 (0.0009) -[2023-10-15 16:50:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14329.1). Total num frames: 105283584. Throughput: 0: 1775.0, 1: 1790.8. Samples: 26334380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:43,441][51532] Avg episode reward: [(0, '51.200'), (1, '45.080')] -[2023-10-15 16:50:43,838][52866] Updated weights for policy 1, policy_version 51490 (0.0009) -[2023-10-15 16:50:44,213][52866] Updated weights for policy 1, policy_version 51500 (0.0008) -[2023-10-15 16:50:44,475][52833] Updated weights for policy 0, policy_version 51330 (0.0007) -[2023-10-15 16:50:44,571][52866] Updated weights for policy 1, policy_version 51510 (0.0007) -[2023-10-15 16:50:44,844][52833] Updated weights for policy 0, policy_version 51340 (0.0007) -[2023-10-15 16:50:44,936][52866] Updated weights for policy 1, policy_version 51520 (0.0008) -[2023-10-15 16:50:45,216][52833] Updated weights for policy 0, policy_version 51350 (0.0010) -[2023-10-15 16:50:45,577][52833] Updated weights for policy 0, policy_version 51360 (0.0008) -[2023-10-15 16:50:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 105349120. Throughput: 0: 1770.2, 1: 1797.7. Samples: 26344128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:48,441][51532] Avg episode reward: [(0, '53.530'), (1, '46.490')] -[2023-10-15 16:50:48,623][52866] Updated weights for policy 1, policy_version 51530 (0.0009) -[2023-10-15 16:50:48,980][52866] Updated weights for policy 1, policy_version 51540 (0.0007) -[2023-10-15 16:50:49,358][52866] Updated weights for policy 1, policy_version 51550 (0.0009) -[2023-10-15 16:50:49,460][52833] Updated weights for policy 0, policy_version 51370 (0.0007) -[2023-10-15 16:50:49,829][52833] Updated weights for policy 0, policy_version 51380 (0.0008) -[2023-10-15 16:50:50,197][52833] Updated weights for policy 0, policy_version 51390 (0.0008) -[2023-10-15 16:50:53,249][52866] Updated weights for policy 1, policy_version 51560 (0.0011) -[2023-10-15 16:50:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 105414656. Throughput: 0: 1760.5, 1: 1797.6. Samples: 26365988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:53,442][51532] Avg episode reward: [(0, '54.910'), (1, '46.250')] -[2023-10-15 16:50:53,619][52866] Updated weights for policy 1, policy_version 51570 (0.0007) -[2023-10-15 16:50:53,936][52833] Updated weights for policy 0, policy_version 51400 (0.0009) -[2023-10-15 16:50:53,989][52866] Updated weights for policy 1, policy_version 51580 (0.0009) -[2023-10-15 16:50:54,314][52833] Updated weights for policy 0, policy_version 51410 (0.0010) -[2023-10-15 16:50:54,681][52833] Updated weights for policy 0, policy_version 51420 (0.0009) -[2023-10-15 16:50:57,735][52866] Updated weights for policy 1, policy_version 51590 (0.0008) -[2023-10-15 16:50:58,108][52866] Updated weights for policy 1, policy_version 51600 (0.0007) -[2023-10-15 16:50:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 105480192. Throughput: 0: 1774.9, 1: 1812.9. Samples: 26387814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:50:58,441][51532] Avg episode reward: [(0, '54.470'), (1, '47.160')] -[2023-10-15 16:50:58,474][52866] Updated weights for policy 1, policy_version 51610 (0.0007) -[2023-10-15 16:50:58,495][52833] Updated weights for policy 0, policy_version 51430 (0.0009) -[2023-10-15 16:50:58,860][52833] Updated weights for policy 0, policy_version 51440 (0.0008) -[2023-10-15 16:50:59,238][52833] Updated weights for policy 0, policy_version 51450 (0.0007) -[2023-10-15 16:51:02,113][52866] Updated weights for policy 1, policy_version 51620 (0.0008) -[2023-10-15 16:51:02,476][52866] Updated weights for policy 1, policy_version 51630 (0.0008) -[2023-10-15 16:51:02,834][52866] Updated weights for policy 1, policy_version 51640 (0.0009) -[2023-10-15 16:51:02,907][52833] Updated weights for policy 0, policy_version 51460 (0.0007) -[2023-10-15 16:51:03,293][52833] Updated weights for policy 0, policy_version 51470 (0.0010) -[2023-10-15 16:51:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 105578496. Throughput: 0: 1757.6, 1: 1800.0. Samples: 26398388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:51:03,441][51532] Avg episode reward: [(0, '52.800'), (1, '50.670')] -[2023-10-15 16:51:03,665][52833] Updated weights for policy 0, policy_version 51480 (0.0009) -[2023-10-15 16:51:06,546][52866] Updated weights for policy 1, policy_version 51650 (0.0008) -[2023-10-15 16:51:06,922][52866] Updated weights for policy 1, policy_version 51660 (0.0009) -[2023-10-15 16:51:07,287][52866] Updated weights for policy 1, policy_version 51670 (0.0009) -[2023-10-15 16:51:07,392][52833] Updated weights for policy 0, policy_version 51490 (0.0008) -[2023-10-15 16:51:07,656][52866] Updated weights for policy 1, policy_version 51680 (0.0010) -[2023-10-15 16:51:07,762][52833] Updated weights for policy 0, policy_version 51500 (0.0009) -[2023-10-15 16:51:08,130][52833] Updated weights for policy 0, policy_version 51510 (0.0010) -[2023-10-15 16:51:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 105644032. Throughput: 0: 1777.2, 1: 1807.0. Samples: 26420320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:51:08,441][51532] Avg episode reward: [(0, '56.910'), (1, '48.210')] -[2023-10-15 16:51:08,496][52833] Updated weights for policy 0, policy_version 51520 (0.0009) -[2023-10-15 16:51:11,340][52866] Updated weights for policy 1, policy_version 51690 (0.0010) -[2023-10-15 16:51:11,705][52866] Updated weights for policy 1, policy_version 51700 (0.0007) -[2023-10-15 16:51:12,075][52866] Updated weights for policy 1, policy_version 51710 (0.0008) -[2023-10-15 16:51:12,401][52833] Updated weights for policy 0, policy_version 51530 (0.0008) -[2023-10-15 16:51:12,772][52833] Updated weights for policy 0, policy_version 51540 (0.0008) -[2023-10-15 16:51:13,153][52833] Updated weights for policy 0, policy_version 51550 (0.0011) -[2023-10-15 16:51:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105742336. Throughput: 0: 1773.6, 1: 1801.1. Samples: 26441128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:51:13,441][51532] Avg episode reward: [(0, '56.560'), (1, '48.980')] -[2023-10-15 16:51:15,691][52866] Updated weights for policy 1, policy_version 51720 (0.0007) -[2023-10-15 16:51:16,060][52866] Updated weights for policy 1, policy_version 51730 (0.0009) -[2023-10-15 16:51:16,426][52866] Updated weights for policy 1, policy_version 51740 (0.0008) -[2023-10-15 16:51:17,042][52833] Updated weights for policy 0, policy_version 51560 (0.0011) -[2023-10-15 16:51:17,414][52833] Updated weights for policy 0, policy_version 51570 (0.0010) -[2023-10-15 16:51:17,774][52833] Updated weights for policy 0, policy_version 51580 (0.0009) -[2023-10-15 16:51:18,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105807872. Throughput: 0: 1775.2, 1: 1810.8. Samples: 26452856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:51:18,442][51532] Avg episode reward: [(0, '55.720'), (1, '49.540')] -[2023-10-15 16:51:20,221][52866] Updated weights for policy 1, policy_version 51750 (0.0009) -[2023-10-15 16:51:20,583][52866] Updated weights for policy 1, policy_version 51760 (0.0008) -[2023-10-15 16:51:20,959][52866] Updated weights for policy 1, policy_version 51770 (0.0008) -[2023-10-15 16:51:21,584][52833] Updated weights for policy 0, policy_version 51590 (0.0009) -[2023-10-15 16:51:21,954][52833] Updated weights for policy 0, policy_version 51600 (0.0008) -[2023-10-15 16:51:22,330][52833] Updated weights for policy 0, policy_version 51610 (0.0008) -[2023-10-15 16:51:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105873408. Throughput: 0: 1789.1, 1: 1804.6. Samples: 26473818. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 16:51:23,442][51532] Avg episode reward: [(0, '55.590'), (1, '53.120')] -[2023-10-15 16:51:24,631][52866] Updated weights for policy 1, policy_version 51780 (0.0009) -[2023-10-15 16:51:24,995][52866] Updated weights for policy 1, policy_version 51790 (0.0009) -[2023-10-15 16:51:25,362][52866] Updated weights for policy 1, policy_version 51800 (0.0008) -[2023-10-15 16:51:25,979][52833] Updated weights for policy 0, policy_version 51620 (0.0010) -[2023-10-15 16:51:26,356][52833] Updated weights for policy 0, policy_version 51630 (0.0008) -[2023-10-15 16:51:26,720][52833] Updated weights for policy 0, policy_version 51640 (0.0008) -[2023-10-15 16:51:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 105938944. Throughput: 0: 1768.4, 1: 1811.6. Samples: 26495478. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 16:51:28,442][51532] Avg episode reward: [(0, '57.740'), (1, '51.600')] -[2023-10-15 16:51:28,451][52410] Saving new best policy, reward=57.740! -[2023-10-15 16:51:29,188][52866] Updated weights for policy 1, policy_version 51810 (0.0009) -[2023-10-15 16:51:29,588][52866] Updated weights for policy 1, policy_version 51820 (0.0007) -[2023-10-15 16:51:29,952][52866] Updated weights for policy 1, policy_version 51830 (0.0008) -[2023-10-15 16:51:30,323][52866] Updated weights for policy 1, policy_version 51840 (0.0009) -[2023-10-15 16:51:30,665][52833] Updated weights for policy 0, policy_version 51650 (0.0008) -[2023-10-15 16:51:31,026][52833] Updated weights for policy 0, policy_version 51660 (0.0008) -[2023-10-15 16:51:31,401][52833] Updated weights for policy 0, policy_version 51670 (0.0009) -[2023-10-15 16:51:31,764][52833] Updated weights for policy 0, policy_version 51680 (0.0007) -[2023-10-15 16:51:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 106004480. Throughput: 0: 1794.6, 1: 1807.3. Samples: 26506212. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 16:51:33,442][51532] Avg episode reward: [(0, '57.460'), (1, '51.270')] -[2023-10-15 16:51:34,032][52866] Updated weights for policy 1, policy_version 51850 (0.0010) -[2023-10-15 16:51:34,395][52866] Updated weights for policy 1, policy_version 51860 (0.0010) -[2023-10-15 16:51:34,768][52866] Updated weights for policy 1, policy_version 51870 (0.0012) -[2023-10-15 16:51:35,642][52833] Updated weights for policy 0, policy_version 51690 (0.0008) -[2023-10-15 16:51:36,000][52833] Updated weights for policy 0, policy_version 51700 (0.0008) -[2023-10-15 16:51:36,369][52833] Updated weights for policy 0, policy_version 51710 (0.0007) -[2023-10-15 16:51:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 106070016. Throughput: 0: 1777.1, 1: 1812.9. Samples: 26527538. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 16:51:38,441][51532] Avg episode reward: [(0, '56.310'), (1, '49.690')] -[2023-10-15 16:51:38,566][52866] Updated weights for policy 1, policy_version 51880 (0.0009) -[2023-10-15 16:51:38,928][52866] Updated weights for policy 1, policy_version 51890 (0.0010) -[2023-10-15 16:51:39,297][52866] Updated weights for policy 1, policy_version 51900 (0.0010) -[2023-10-15 16:51:40,108][52833] Updated weights for policy 0, policy_version 51720 (0.0009) -[2023-10-15 16:51:40,480][52833] Updated weights for policy 0, policy_version 51730 (0.0009) -[2023-10-15 16:51:40,854][52833] Updated weights for policy 0, policy_version 51740 (0.0008) -[2023-10-15 16:51:43,077][52866] Updated weights for policy 1, policy_version 51910 (0.0009) -[2023-10-15 16:51:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 106135552. Throughput: 0: 1782.9, 1: 1814.1. Samples: 26549680. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 16:51:43,442][51532] Avg episode reward: [(0, '58.920'), (1, '49.910')] -[2023-10-15 16:51:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000051744_52985856.pth... -[2023-10-15 16:51:43,454][52866] Updated weights for policy 1, policy_version 51920 (0.0009) -[2023-10-15 16:51:43,490][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000050080_51281920.pth -[2023-10-15 16:51:43,495][52410] Saving new best policy, reward=58.920! -[2023-10-15 16:51:43,816][52866] Updated weights for policy 1, policy_version 51930 (0.0008) -[2023-10-15 16:51:44,035][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000051936_53182464.pth... -[2023-10-15 16:51:44,064][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000050240_51445760.pth -[2023-10-15 16:51:44,585][52833] Updated weights for policy 0, policy_version 51750 (0.0009) -[2023-10-15 16:51:44,959][52833] Updated weights for policy 0, policy_version 51760 (0.0009) -[2023-10-15 16:51:45,339][52833] Updated weights for policy 0, policy_version 51770 (0.0007) -[2023-10-15 16:51:47,435][52866] Updated weights for policy 1, policy_version 51940 (0.0008) -[2023-10-15 16:51:47,809][52866] Updated weights for policy 1, policy_version 51950 (0.0008) -[2023-10-15 16:51:48,169][52866] Updated weights for policy 1, policy_version 51960 (0.0007) -[2023-10-15 16:51:48,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 106201088. Throughput: 0: 1780.0, 1: 1803.0. Samples: 26559626. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 16:51:48,442][51532] Avg episode reward: [(0, '58.540'), (1, '50.640')] -[2023-10-15 16:51:49,034][52833] Updated weights for policy 0, policy_version 51780 (0.0009) -[2023-10-15 16:51:49,397][52833] Updated weights for policy 0, policy_version 51790 (0.0008) -[2023-10-15 16:51:49,765][52833] Updated weights for policy 0, policy_version 51800 (0.0009) -[2023-10-15 16:51:51,848][52866] Updated weights for policy 1, policy_version 51970 (0.0008) -[2023-10-15 16:51:52,210][52866] Updated weights for policy 1, policy_version 51980 (0.0008) -[2023-10-15 16:51:52,572][52866] Updated weights for policy 1, policy_version 51990 (0.0007) -[2023-10-15 16:51:52,943][52866] Updated weights for policy 1, policy_version 52000 (0.0009) -[2023-10-15 16:51:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 106299392. Throughput: 0: 1779.6, 1: 1811.9. Samples: 26581934. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 16:51:53,442][51532] Avg episode reward: [(0, '59.930'), (1, '49.390')] -[2023-10-15 16:51:53,485][52833] Updated weights for policy 0, policy_version 51810 (0.0009) -[2023-10-15 16:51:53,896][52833] Updated weights for policy 0, policy_version 51820 (0.0009) -[2023-10-15 16:51:54,273][52833] Updated weights for policy 0, policy_version 51830 (0.0009) -[2023-10-15 16:51:54,636][52410] Saving new best policy, reward=59.930! -[2023-10-15 16:51:54,639][52833] Updated weights for policy 0, policy_version 51840 (0.0009) -[2023-10-15 16:51:56,672][52866] Updated weights for policy 1, policy_version 52010 (0.0008) -[2023-10-15 16:51:57,033][52866] Updated weights for policy 1, policy_version 52020 (0.0007) -[2023-10-15 16:51:57,396][52866] Updated weights for policy 1, policy_version 52030 (0.0009) -[2023-10-15 16:51:58,214][52833] Updated weights for policy 0, policy_version 51850 (0.0011) -[2023-10-15 16:51:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 106364928. Throughput: 0: 1808.7, 1: 1803.0. Samples: 26603652. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 16:51:58,441][51532] Avg episode reward: [(0, '57.150'), (1, '48.140')] -[2023-10-15 16:51:58,585][52833] Updated weights for policy 0, policy_version 51860 (0.0010) -[2023-10-15 16:51:58,945][52833] Updated weights for policy 0, policy_version 51870 (0.0011) -[2023-10-15 16:52:01,103][52866] Updated weights for policy 1, policy_version 52040 (0.0007) -[2023-10-15 16:52:01,464][52866] Updated weights for policy 1, policy_version 52050 (0.0009) -[2023-10-15 16:52:01,833][52866] Updated weights for policy 1, policy_version 52060 (0.0009) -[2023-10-15 16:52:02,920][52833] Updated weights for policy 0, policy_version 51880 (0.0010) -[2023-10-15 16:52:03,287][52833] Updated weights for policy 0, policy_version 51890 (0.0007) -[2023-10-15 16:52:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 106430464. Throughput: 0: 1781.4, 1: 1813.6. Samples: 26614628. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 16:52:03,441][51532] Avg episode reward: [(0, '56.770'), (1, '49.670')] -[2023-10-15 16:52:03,656][52833] Updated weights for policy 0, policy_version 51900 (0.0010) -[2023-10-15 16:52:05,568][52866] Updated weights for policy 1, policy_version 52070 (0.0009) -[2023-10-15 16:52:05,941][52866] Updated weights for policy 1, policy_version 52080 (0.0008) -[2023-10-15 16:52:06,301][52866] Updated weights for policy 1, policy_version 52090 (0.0011) -[2023-10-15 16:52:07,393][52833] Updated weights for policy 0, policy_version 51910 (0.0009) -[2023-10-15 16:52:07,761][52833] Updated weights for policy 0, policy_version 51920 (0.0007) -[2023-10-15 16:52:08,140][52833] Updated weights for policy 0, policy_version 51930 (0.0007) -[2023-10-15 16:52:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 106528768. Throughput: 0: 1797.5, 1: 1804.2. Samples: 26635896. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 16:52:08,442][51532] Avg episode reward: [(0, '55.510'), (1, '47.820')] -[2023-10-15 16:52:09,926][52866] Updated weights for policy 1, policy_version 52100 (0.0009) -[2023-10-15 16:52:10,293][52866] Updated weights for policy 1, policy_version 52110 (0.0008) -[2023-10-15 16:52:10,660][52866] Updated weights for policy 1, policy_version 52120 (0.0007) -[2023-10-15 16:52:11,989][52833] Updated weights for policy 0, policy_version 51940 (0.0008) -[2023-10-15 16:52:12,353][52833] Updated weights for policy 0, policy_version 51950 (0.0009) -[2023-10-15 16:52:12,721][52833] Updated weights for policy 0, policy_version 51960 (0.0009) -[2023-10-15 16:52:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106594304. Throughput: 0: 1792.8, 1: 1807.5. Samples: 26657492. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 16:52:13,441][51532] Avg episode reward: [(0, '55.610'), (1, '46.350')] -[2023-10-15 16:52:14,485][52866] Updated weights for policy 1, policy_version 52130 (0.0009) -[2023-10-15 16:52:14,901][52866] Updated weights for policy 1, policy_version 52140 (0.0008) -[2023-10-15 16:52:15,264][52866] Updated weights for policy 1, policy_version 52150 (0.0010) -[2023-10-15 16:52:15,622][52866] Updated weights for policy 1, policy_version 52160 (0.0010) -[2023-10-15 16:52:16,212][52833] Updated weights for policy 0, policy_version 51970 (0.0007) -[2023-10-15 16:52:16,583][52833] Updated weights for policy 0, policy_version 51980 (0.0009) -[2023-10-15 16:52:16,943][52833] Updated weights for policy 0, policy_version 51990 (0.0008) -[2023-10-15 16:52:17,320][52833] Updated weights for policy 0, policy_version 52000 (0.0009) -[2023-10-15 16:52:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106659840. Throughput: 0: 1802.6, 1: 1808.0. Samples: 26668686. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 16:52:18,441][51532] Avg episode reward: [(0, '51.010'), (1, '48.110')] -[2023-10-15 16:52:19,290][52866] Updated weights for policy 1, policy_version 52170 (0.0011) -[2023-10-15 16:52:19,661][52866] Updated weights for policy 1, policy_version 52180 (0.0009) -[2023-10-15 16:52:20,042][52866] Updated weights for policy 1, policy_version 52190 (0.0010) -[2023-10-15 16:52:21,082][52833] Updated weights for policy 0, policy_version 52010 (0.0007) -[2023-10-15 16:52:21,447][52833] Updated weights for policy 0, policy_version 52020 (0.0010) -[2023-10-15 16:52:21,815][52833] Updated weights for policy 0, policy_version 52030 (0.0009) -[2023-10-15 16:52:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 106725376. Throughput: 0: 1800.4, 1: 1807.2. Samples: 26689878. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 16:52:23,441][51532] Avg episode reward: [(0, '49.620'), (1, '47.570')] -[2023-10-15 16:52:23,726][52866] Updated weights for policy 1, policy_version 52200 (0.0008) -[2023-10-15 16:52:24,087][52866] Updated weights for policy 1, policy_version 52210 (0.0011) -[2023-10-15 16:52:24,456][52866] Updated weights for policy 1, policy_version 52220 (0.0008) -[2023-10-15 16:52:25,602][52833] Updated weights for policy 0, policy_version 52040 (0.0007) -[2023-10-15 16:52:25,975][52833] Updated weights for policy 0, policy_version 52050 (0.0007) -[2023-10-15 16:52:26,347][52833] Updated weights for policy 0, policy_version 52060 (0.0010) -[2023-10-15 16:52:28,177][52866] Updated weights for policy 1, policy_version 52230 (0.0007) -[2023-10-15 16:52:28,441][51532] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 106790912. Throughput: 0: 1800.9, 1: 1813.3. Samples: 26712322. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 16:52:28,442][51532] Avg episode reward: [(0, '48.470'), (1, '48.120')] -[2023-10-15 16:52:28,532][52866] Updated weights for policy 1, policy_version 52240 (0.0010) -[2023-10-15 16:52:28,898][52866] Updated weights for policy 1, policy_version 52250 (0.0009) -[2023-10-15 16:52:29,891][52833] Updated weights for policy 0, policy_version 52070 (0.0011) -[2023-10-15 16:52:30,258][52833] Updated weights for policy 0, policy_version 52080 (0.0009) -[2023-10-15 16:52:30,629][52833] Updated weights for policy 0, policy_version 52090 (0.0009) -[2023-10-15 16:52:32,669][52866] Updated weights for policy 1, policy_version 52260 (0.0009) -[2023-10-15 16:52:33,030][52866] Updated weights for policy 1, policy_version 52270 (0.0008) -[2023-10-15 16:52:33,397][52866] Updated weights for policy 1, policy_version 52280 (0.0007) -[2023-10-15 16:52:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 106856448. Throughput: 0: 1804.2, 1: 1813.7. Samples: 26722432. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 16:52:33,442][51532] Avg episode reward: [(0, '47.420'), (1, '48.970')] -[2023-10-15 16:52:34,235][52833] Updated weights for policy 0, policy_version 52100 (0.0010) -[2023-10-15 16:52:34,601][52833] Updated weights for policy 0, policy_version 52110 (0.0009) -[2023-10-15 16:52:34,982][52833] Updated weights for policy 0, policy_version 52120 (0.0009) -[2023-10-15 16:52:37,035][52866] Updated weights for policy 1, policy_version 52290 (0.0008) -[2023-10-15 16:52:37,403][52866] Updated weights for policy 1, policy_version 52300 (0.0011) -[2023-10-15 16:52:37,773][52866] Updated weights for policy 1, policy_version 52310 (0.0007) -[2023-10-15 16:52:38,139][52866] Updated weights for policy 1, policy_version 52320 (0.0008) -[2023-10-15 16:52:38,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 106954752. Throughput: 0: 1798.8, 1: 1819.5. Samples: 26744758. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:52:38,442][51532] Avg episode reward: [(0, '45.430'), (1, '45.690')] -[2023-10-15 16:52:38,833][52833] Updated weights for policy 0, policy_version 52130 (0.0007) -[2023-10-15 16:52:39,235][52833] Updated weights for policy 0, policy_version 52140 (0.0009) -[2023-10-15 16:52:39,602][52833] Updated weights for policy 0, policy_version 52150 (0.0008) -[2023-10-15 16:52:39,975][52833] Updated weights for policy 0, policy_version 52160 (0.0007) -[2023-10-15 16:52:41,864][52866] Updated weights for policy 1, policy_version 52330 (0.0009) -[2023-10-15 16:52:42,233][52866] Updated weights for policy 1, policy_version 52340 (0.0007) -[2023-10-15 16:52:42,603][52866] Updated weights for policy 1, policy_version 52350 (0.0010) -[2023-10-15 16:52:43,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 107020288. Throughput: 0: 1794.9, 1: 1808.4. Samples: 26765802. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:52:43,441][51532] Avg episode reward: [(0, '45.950'), (1, '46.100')] -[2023-10-15 16:52:43,668][52833] Updated weights for policy 0, policy_version 52170 (0.0008) -[2023-10-15 16:52:44,044][52833] Updated weights for policy 0, policy_version 52180 (0.0007) -[2023-10-15 16:52:44,420][52833] Updated weights for policy 0, policy_version 52190 (0.0007) -[2023-10-15 16:52:46,317][52866] Updated weights for policy 1, policy_version 52360 (0.0010) -[2023-10-15 16:52:46,685][52866] Updated weights for policy 1, policy_version 52370 (0.0008) -[2023-10-15 16:52:47,052][52866] Updated weights for policy 1, policy_version 52380 (0.0008) -[2023-10-15 16:52:48,132][52833] Updated weights for policy 0, policy_version 52200 (0.0009) -[2023-10-15 16:52:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 107085824. Throughput: 0: 1799.9, 1: 1811.9. Samples: 26777160. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:52:48,441][51532] Avg episode reward: [(0, '47.170'), (1, '46.100')] -[2023-10-15 16:52:48,494][52833] Updated weights for policy 0, policy_version 52210 (0.0009) -[2023-10-15 16:52:48,868][52833] Updated weights for policy 0, policy_version 52220 (0.0010) -[2023-10-15 16:52:50,887][52866] Updated weights for policy 1, policy_version 52390 (0.0007) -[2023-10-15 16:52:51,255][52866] Updated weights for policy 1, policy_version 52400 (0.0010) -[2023-10-15 16:52:51,631][52866] Updated weights for policy 1, policy_version 52410 (0.0009) -[2023-10-15 16:52:52,587][52833] Updated weights for policy 0, policy_version 52230 (0.0010) -[2023-10-15 16:52:52,958][52833] Updated weights for policy 0, policy_version 52240 (0.0008) -[2023-10-15 16:52:53,329][52833] Updated weights for policy 0, policy_version 52250 (0.0011) -[2023-10-15 16:52:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 107151360. Throughput: 0: 1801.4, 1: 1799.8. Samples: 26797952. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:52:53,441][51532] Avg episode reward: [(0, '45.580'), (1, '46.080')] -[2023-10-15 16:52:55,206][52866] Updated weights for policy 1, policy_version 52420 (0.0008) -[2023-10-15 16:52:55,573][52866] Updated weights for policy 1, policy_version 52430 (0.0008) -[2023-10-15 16:52:55,945][52866] Updated weights for policy 1, policy_version 52440 (0.0008) -[2023-10-15 16:52:57,114][52833] Updated weights for policy 0, policy_version 52260 (0.0011) -[2023-10-15 16:52:57,494][52833] Updated weights for policy 0, policy_version 52270 (0.0010) -[2023-10-15 16:52:57,859][52833] Updated weights for policy 0, policy_version 52280 (0.0011) -[2023-10-15 16:52:58,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 107249664. Throughput: 0: 1806.5, 1: 1802.7. Samples: 26819908. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:52:58,442][51532] Avg episode reward: [(0, '47.210'), (1, '47.060')] -[2023-10-15 16:52:59,616][52866] Updated weights for policy 1, policy_version 52450 (0.0008) -[2023-10-15 16:53:00,003][52866] Updated weights for policy 1, policy_version 52460 (0.0007) -[2023-10-15 16:53:00,362][52866] Updated weights for policy 1, policy_version 52470 (0.0008) -[2023-10-15 16:53:00,730][52866] Updated weights for policy 1, policy_version 52480 (0.0008) -[2023-10-15 16:53:01,689][52833] Updated weights for policy 0, policy_version 52290 (0.0010) -[2023-10-15 16:53:02,059][52833] Updated weights for policy 0, policy_version 52300 (0.0010) -[2023-10-15 16:53:02,435][52833] Updated weights for policy 0, policy_version 52310 (0.0008) -[2023-10-15 16:53:02,802][52833] Updated weights for policy 0, policy_version 52320 (0.0007) -[2023-10-15 16:53:03,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 107315200. Throughput: 0: 1795.0, 1: 1808.7. Samples: 26830850. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:53:03,442][51532] Avg episode reward: [(0, '48.230'), (1, '46.470')] -[2023-10-15 16:53:04,398][52866] Updated weights for policy 1, policy_version 52490 (0.0009) -[2023-10-15 16:53:04,772][52866] Updated weights for policy 1, policy_version 52500 (0.0009) -[2023-10-15 16:53:05,135][52866] Updated weights for policy 1, policy_version 52510 (0.0008) -[2023-10-15 16:53:06,614][52833] Updated weights for policy 0, policy_version 52330 (0.0009) -[2023-10-15 16:53:06,993][52833] Updated weights for policy 0, policy_version 52340 (0.0008) -[2023-10-15 16:53:07,354][52833] Updated weights for policy 0, policy_version 52350 (0.0008) -[2023-10-15 16:53:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 107380736. Throughput: 0: 1806.9, 1: 1810.4. Samples: 26852656. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-15 16:53:08,441][51532] Avg episode reward: [(0, '50.120'), (1, '48.180')] -[2023-10-15 16:53:08,880][52866] Updated weights for policy 1, policy_version 52520 (0.0010) -[2023-10-15 16:53:09,250][52866] Updated weights for policy 1, policy_version 52530 (0.0008) -[2023-10-15 16:53:09,621][52866] Updated weights for policy 1, policy_version 52540 (0.0008) -[2023-10-15 16:53:11,113][52833] Updated weights for policy 0, policy_version 52360 (0.0009) -[2023-10-15 16:53:11,486][52833] Updated weights for policy 0, policy_version 52370 (0.0009) -[2023-10-15 16:53:11,860][52833] Updated weights for policy 0, policy_version 52380 (0.0009) -[2023-10-15 16:53:13,403][52866] Updated weights for policy 1, policy_version 52550 (0.0008) -[2023-10-15 16:53:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107446272. Throughput: 0: 1791.1, 1: 1811.8. Samples: 26874452. Policy #0 lag: (min: 17.0, avg: 37.8, max: 49.0) -[2023-10-15 16:53:13,441][51532] Avg episode reward: [(0, '49.600'), (1, '48.650')] -[2023-10-15 16:53:13,774][52866] Updated weights for policy 1, policy_version 52560 (0.0009) -[2023-10-15 16:53:14,152][52866] Updated weights for policy 1, policy_version 52570 (0.0011) -[2023-10-15 16:53:15,443][52833] Updated weights for policy 0, policy_version 52390 (0.0007) -[2023-10-15 16:53:15,798][52833] Updated weights for policy 0, policy_version 52400 (0.0008) -[2023-10-15 16:53:16,184][52833] Updated weights for policy 0, policy_version 52410 (0.0011) -[2023-10-15 16:53:17,987][52866] Updated weights for policy 1, policy_version 52580 (0.0009) -[2023-10-15 16:53:18,350][52866] Updated weights for policy 1, policy_version 52590 (0.0008) -[2023-10-15 16:53:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 107511808. Throughput: 0: 1806.1, 1: 1807.4. Samples: 26885042. Policy #0 lag: (min: 17.0, avg: 37.8, max: 49.0) -[2023-10-15 16:53:18,442][51532] Avg episode reward: [(0, '50.780'), (1, '47.560')] -[2023-10-15 16:53:18,724][52866] Updated weights for policy 1, policy_version 52600 (0.0008) -[2023-10-15 16:53:19,922][52833] Updated weights for policy 0, policy_version 52420 (0.0009) -[2023-10-15 16:53:20,283][52833] Updated weights for policy 0, policy_version 52430 (0.0009) -[2023-10-15 16:53:20,650][52833] Updated weights for policy 0, policy_version 52440 (0.0009) -[2023-10-15 16:53:22,557][52866] Updated weights for policy 1, policy_version 52610 (0.0009) -[2023-10-15 16:53:22,930][52866] Updated weights for policy 1, policy_version 52620 (0.0007) -[2023-10-15 16:53:23,293][52866] Updated weights for policy 1, policy_version 52630 (0.0008) -[2023-10-15 16:53:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 107577344. Throughput: 0: 1794.0, 1: 1796.1. Samples: 26906308. Policy #0 lag: (min: 17.0, avg: 37.8, max: 49.0) -[2023-10-15 16:53:23,441][51532] Avg episode reward: [(0, '55.070'), (1, '48.100')] -[2023-10-15 16:53:23,663][52866] Updated weights for policy 1, policy_version 52640 (0.0009) -[2023-10-15 16:53:24,608][52833] Updated weights for policy 0, policy_version 52450 (0.0010) -[2023-10-15 16:53:25,002][52833] Updated weights for policy 0, policy_version 52460 (0.0009) -[2023-10-15 16:53:25,376][52833] Updated weights for policy 0, policy_version 52470 (0.0007) -[2023-10-15 16:53:25,737][52833] Updated weights for policy 0, policy_version 52480 (0.0007) -[2023-10-15 16:53:27,315][52866] Updated weights for policy 1, policy_version 52650 (0.0007) -[2023-10-15 16:53:27,682][52866] Updated weights for policy 1, policy_version 52660 (0.0008) -[2023-10-15 16:53:28,046][52866] Updated weights for policy 1, policy_version 52670 (0.0007) -[2023-10-15 16:53:28,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 107675648. Throughput: 0: 1793.6, 1: 1802.0. Samples: 26927604. Policy #0 lag: (min: 17.0, avg: 37.8, max: 49.0) -[2023-10-15 16:53:28,441][51532] Avg episode reward: [(0, '54.800'), (1, '49.990')] -[2023-10-15 16:53:29,404][52833] Updated weights for policy 0, policy_version 52490 (0.0008) -[2023-10-15 16:53:29,781][52833] Updated weights for policy 0, policy_version 52500 (0.0009) -[2023-10-15 16:53:30,142][52833] Updated weights for policy 0, policy_version 52510 (0.0008) -[2023-10-15 16:53:31,747][52866] Updated weights for policy 1, policy_version 52680 (0.0008) -[2023-10-15 16:53:32,111][52866] Updated weights for policy 1, policy_version 52690 (0.0007) -[2023-10-15 16:53:32,474][52866] Updated weights for policy 1, policy_version 52700 (0.0008) -[2023-10-15 16:53:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 107741184. Throughput: 0: 1796.4, 1: 1799.1. Samples: 26938960. Policy #0 lag: (min: 17.0, avg: 37.8, max: 49.0) -[2023-10-15 16:53:33,442][51532] Avg episode reward: [(0, '53.250'), (1, '50.610')] -[2023-10-15 16:53:33,870][52833] Updated weights for policy 0, policy_version 52520 (0.0007) -[2023-10-15 16:53:34,252][52833] Updated weights for policy 0, policy_version 52530 (0.0009) -[2023-10-15 16:53:34,616][52833] Updated weights for policy 0, policy_version 52540 (0.0011) -[2023-10-15 16:53:36,266][52866] Updated weights for policy 1, policy_version 52710 (0.0010) -[2023-10-15 16:53:36,633][52866] Updated weights for policy 1, policy_version 52720 (0.0007) -[2023-10-15 16:53:36,996][52866] Updated weights for policy 1, policy_version 52730 (0.0010) -[2023-10-15 16:53:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 107806720. Throughput: 0: 1794.1, 1: 1810.5. Samples: 26960160. Policy #0 lag: (min: 17.0, avg: 37.8, max: 49.0) -[2023-10-15 16:53:38,441][51532] Avg episode reward: [(0, '53.500'), (1, '51.130')] -[2023-10-15 16:53:38,456][52833] Updated weights for policy 0, policy_version 52550 (0.0009) -[2023-10-15 16:53:38,820][52833] Updated weights for policy 0, policy_version 52560 (0.0007) -[2023-10-15 16:53:39,188][52833] Updated weights for policy 0, policy_version 52570 (0.0008) -[2023-10-15 16:53:40,781][52866] Updated weights for policy 1, policy_version 52740 (0.0008) -[2023-10-15 16:53:41,144][52866] Updated weights for policy 1, policy_version 52750 (0.0009) -[2023-10-15 16:53:41,512][52866] Updated weights for policy 1, policy_version 52760 (0.0009) -[2023-10-15 16:53:43,024][52833] Updated weights for policy 0, policy_version 52580 (0.0009) -[2023-10-15 16:53:43,387][52833] Updated weights for policy 0, policy_version 52590 (0.0009) -[2023-10-15 16:53:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 107872256. Throughput: 0: 1813.4, 1: 1790.8. Samples: 26982096. Policy #0 lag: (min: 17.0, avg: 37.8, max: 49.0) -[2023-10-15 16:53:43,441][51532] Avg episode reward: [(0, '55.810'), (1, '51.250')] -[2023-10-15 16:53:43,448][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000052768_54034432.pth... -[2023-10-15 16:53:43,482][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000051072_52297728.pth -[2023-10-15 16:53:43,757][52833] Updated weights for policy 0, policy_version 52600 (0.0008) -[2023-10-15 16:53:44,047][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000052608_53870592.pth... -[2023-10-15 16:53:44,085][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000050912_52133888.pth -[2023-10-15 16:53:45,139][52866] Updated weights for policy 1, policy_version 52770 (0.0008) -[2023-10-15 16:53:45,538][52866] Updated weights for policy 1, policy_version 52780 (0.0010) -[2023-10-15 16:53:45,900][52866] Updated weights for policy 1, policy_version 52790 (0.0009) -[2023-10-15 16:53:46,263][52866] Updated weights for policy 1, policy_version 52800 (0.0007) -[2023-10-15 16:53:47,447][52833] Updated weights for policy 0, policy_version 52610 (0.0008) -[2023-10-15 16:53:47,816][52833] Updated weights for policy 0, policy_version 52620 (0.0009) -[2023-10-15 16:53:48,190][52833] Updated weights for policy 0, policy_version 52630 (0.0008) -[2023-10-15 16:53:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 107937792. Throughput: 0: 1790.6, 1: 1797.6. Samples: 26992316. Policy #0 lag: (min: 17.0, avg: 37.8, max: 49.0) -[2023-10-15 16:53:48,442][51532] Avg episode reward: [(0, '59.050'), (1, '50.890')] -[2023-10-15 16:53:48,568][52833] Updated weights for policy 0, policy_version 52640 (0.0011) -[2023-10-15 16:53:49,959][52866] Updated weights for policy 1, policy_version 52810 (0.0007) -[2023-10-15 16:53:50,328][52866] Updated weights for policy 1, policy_version 52820 (0.0010) -[2023-10-15 16:53:50,688][52866] Updated weights for policy 1, policy_version 52830 (0.0010) -[2023-10-15 16:53:52,328][52833] Updated weights for policy 0, policy_version 52650 (0.0007) -[2023-10-15 16:53:52,705][52833] Updated weights for policy 0, policy_version 52660 (0.0008) -[2023-10-15 16:53:53,077][52833] Updated weights for policy 0, policy_version 52670 (0.0008) -[2023-10-15 16:53:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 108036096. Throughput: 0: 1807.9, 1: 1784.3. Samples: 27014304. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:53:53,442][51532] Avg episode reward: [(0, '58.100'), (1, '51.770')] -[2023-10-15 16:53:54,411][52866] Updated weights for policy 1, policy_version 52840 (0.0008) -[2023-10-15 16:53:54,774][52866] Updated weights for policy 1, policy_version 52850 (0.0008) -[2023-10-15 16:53:55,147][52866] Updated weights for policy 1, policy_version 52860 (0.0007) -[2023-10-15 16:53:56,767][52833] Updated weights for policy 0, policy_version 52680 (0.0007) -[2023-10-15 16:53:57,137][52833] Updated weights for policy 0, policy_version 52690 (0.0007) -[2023-10-15 16:53:57,510][52833] Updated weights for policy 0, policy_version 52700 (0.0008) -[2023-10-15 16:53:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108101632. Throughput: 0: 1789.5, 1: 1792.5. Samples: 27035642. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:53:58,442][51532] Avg episode reward: [(0, '56.440'), (1, '50.800')] -[2023-10-15 16:53:58,909][52866] Updated weights for policy 1, policy_version 52870 (0.0008) -[2023-10-15 16:53:59,270][52866] Updated weights for policy 1, policy_version 52880 (0.0007) -[2023-10-15 16:53:59,635][52866] Updated weights for policy 1, policy_version 52890 (0.0008) -[2023-10-15 16:54:01,180][52833] Updated weights for policy 0, policy_version 52710 (0.0009) -[2023-10-15 16:54:01,548][52833] Updated weights for policy 0, policy_version 52720 (0.0009) -[2023-10-15 16:54:01,920][52833] Updated weights for policy 0, policy_version 52730 (0.0008) -[2023-10-15 16:54:03,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108167168. Throughput: 0: 1810.1, 1: 1790.3. Samples: 27047060. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:54:03,441][51532] Avg episode reward: [(0, '57.920'), (1, '52.220')] -[2023-10-15 16:54:03,570][52866] Updated weights for policy 1, policy_version 52900 (0.0008) -[2023-10-15 16:54:03,933][52866] Updated weights for policy 1, policy_version 52910 (0.0009) -[2023-10-15 16:54:04,303][52866] Updated weights for policy 1, policy_version 52920 (0.0009) -[2023-10-15 16:54:05,509][52833] Updated weights for policy 0, policy_version 52740 (0.0009) -[2023-10-15 16:54:05,866][52833] Updated weights for policy 0, policy_version 52750 (0.0009) -[2023-10-15 16:54:06,240][52833] Updated weights for policy 0, policy_version 52760 (0.0008) -[2023-10-15 16:54:07,925][52866] Updated weights for policy 1, policy_version 52930 (0.0008) -[2023-10-15 16:54:08,295][52866] Updated weights for policy 1, policy_version 52940 (0.0008) -[2023-10-15 16:54:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 108232704. Throughput: 0: 1792.5, 1: 1800.3. Samples: 27067988. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:54:08,442][51532] Avg episode reward: [(0, '54.590'), (1, '51.260')] -[2023-10-15 16:54:08,670][52866] Updated weights for policy 1, policy_version 52950 (0.0007) -[2023-10-15 16:54:09,040][52866] Updated weights for policy 1, policy_version 52960 (0.0009) -[2023-10-15 16:54:09,946][52833] Updated weights for policy 0, policy_version 52770 (0.0007) -[2023-10-15 16:54:10,312][52833] Updated weights for policy 0, policy_version 52780 (0.0007) -[2023-10-15 16:54:10,682][52833] Updated weights for policy 0, policy_version 52790 (0.0007) -[2023-10-15 16:54:11,050][52833] Updated weights for policy 0, policy_version 52800 (0.0008) -[2023-10-15 16:54:12,743][52866] Updated weights for policy 1, policy_version 52970 (0.0007) -[2023-10-15 16:54:13,105][52866] Updated weights for policy 1, policy_version 52980 (0.0007) -[2023-10-15 16:54:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 108298240. Throughput: 0: 1795.8, 1: 1813.4. Samples: 27090020. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:54:13,441][51532] Avg episode reward: [(0, '53.510'), (1, '53.030')] -[2023-10-15 16:54:13,475][52866] Updated weights for policy 1, policy_version 52990 (0.0007) -[2023-10-15 16:54:14,792][52833] Updated weights for policy 0, policy_version 52810 (0.0010) -[2023-10-15 16:54:15,155][52833] Updated weights for policy 0, policy_version 52820 (0.0007) -[2023-10-15 16:54:15,516][52833] Updated weights for policy 0, policy_version 52830 (0.0008) -[2023-10-15 16:54:17,295][52866] Updated weights for policy 1, policy_version 53000 (0.0007) -[2023-10-15 16:54:17,664][52866] Updated weights for policy 1, policy_version 53010 (0.0008) -[2023-10-15 16:54:18,028][52866] Updated weights for policy 1, policy_version 53020 (0.0009) -[2023-10-15 16:54:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 108396544. Throughput: 0: 1790.2, 1: 1798.3. Samples: 27100440. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:54:18,442][51532] Avg episode reward: [(0, '56.180'), (1, '53.850')] -[2023-10-15 16:54:19,286][52833] Updated weights for policy 0, policy_version 52840 (0.0007) -[2023-10-15 16:54:19,659][52833] Updated weights for policy 0, policy_version 52850 (0.0009) -[2023-10-15 16:54:20,033][52833] Updated weights for policy 0, policy_version 52860 (0.0011) -[2023-10-15 16:54:21,825][52866] Updated weights for policy 1, policy_version 53030 (0.0008) -[2023-10-15 16:54:22,194][52866] Updated weights for policy 1, policy_version 53040 (0.0009) -[2023-10-15 16:54:22,553][52866] Updated weights for policy 1, policy_version 53050 (0.0008) -[2023-10-15 16:54:23,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 108462080. Throughput: 0: 1792.8, 1: 1815.2. Samples: 27122520. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 16:54:23,442][51532] Avg episode reward: [(0, '57.950'), (1, '51.460')] -[2023-10-15 16:54:23,665][52833] Updated weights for policy 0, policy_version 52870 (0.0009) -[2023-10-15 16:54:24,038][52833] Updated weights for policy 0, policy_version 52880 (0.0010) -[2023-10-15 16:54:24,422][52833] Updated weights for policy 0, policy_version 52890 (0.0009) -[2023-10-15 16:54:26,224][52866] Updated weights for policy 1, policy_version 53060 (0.0008) -[2023-10-15 16:54:26,592][52866] Updated weights for policy 1, policy_version 53070 (0.0007) -[2023-10-15 16:54:26,961][52866] Updated weights for policy 1, policy_version 53080 (0.0008) -[2023-10-15 16:54:28,155][52833] Updated weights for policy 0, policy_version 52900 (0.0010) -[2023-10-15 16:54:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 108527616. Throughput: 0: 1800.2, 1: 1801.6. Samples: 27144178. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 16:54:28,442][51532] Avg episode reward: [(0, '54.970'), (1, '51.530')] -[2023-10-15 16:54:28,525][52833] Updated weights for policy 0, policy_version 52910 (0.0007) -[2023-10-15 16:54:28,886][52833] Updated weights for policy 0, policy_version 52920 (0.0007) -[2023-10-15 16:54:30,677][52866] Updated weights for policy 1, policy_version 53090 (0.0009) -[2023-10-15 16:54:31,079][52866] Updated weights for policy 1, policy_version 53100 (0.0007) -[2023-10-15 16:54:31,432][52866] Updated weights for policy 1, policy_version 53110 (0.0007) -[2023-10-15 16:54:31,802][52866] Updated weights for policy 1, policy_version 53120 (0.0008) -[2023-10-15 16:54:32,630][52833] Updated weights for policy 0, policy_version 52930 (0.0007) -[2023-10-15 16:54:33,003][52833] Updated weights for policy 0, policy_version 52940 (0.0011) -[2023-10-15 16:54:33,366][52833] Updated weights for policy 0, policy_version 52950 (0.0008) -[2023-10-15 16:54:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 108593152. Throughput: 0: 1800.9, 1: 1817.2. Samples: 27155130. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 16:54:33,441][51532] Avg episode reward: [(0, '53.790'), (1, '51.600')] -[2023-10-15 16:54:33,733][52833] Updated weights for policy 0, policy_version 52960 (0.0010) -[2023-10-15 16:54:35,511][52866] Updated weights for policy 1, policy_version 53130 (0.0009) -[2023-10-15 16:54:35,879][52866] Updated weights for policy 1, policy_version 53140 (0.0009) -[2023-10-15 16:54:36,258][52866] Updated weights for policy 1, policy_version 53150 (0.0009) -[2023-10-15 16:54:37,525][52833] Updated weights for policy 0, policy_version 52970 (0.0008) -[2023-10-15 16:54:37,891][52833] Updated weights for policy 0, policy_version 52980 (0.0009) -[2023-10-15 16:54:38,265][52833] Updated weights for policy 0, policy_version 52990 (0.0008) -[2023-10-15 16:54:38,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 108691456. Throughput: 0: 1807.4, 1: 1803.6. Samples: 27176798. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 16:54:38,442][51532] Avg episode reward: [(0, '53.020'), (1, '50.240')] -[2023-10-15 16:54:39,881][52866] Updated weights for policy 1, policy_version 53160 (0.0009) -[2023-10-15 16:54:40,244][52866] Updated weights for policy 1, policy_version 53170 (0.0009) -[2023-10-15 16:54:40,615][52866] Updated weights for policy 1, policy_version 53180 (0.0009) -[2023-10-15 16:54:41,976][52833] Updated weights for policy 0, policy_version 53000 (0.0009) -[2023-10-15 16:54:42,353][52833] Updated weights for policy 0, policy_version 53010 (0.0009) -[2023-10-15 16:54:42,716][52833] Updated weights for policy 0, policy_version 53020 (0.0007) -[2023-10-15 16:54:43,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 108756992. Throughput: 0: 1809.0, 1: 1802.5. Samples: 27198160. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 16:54:43,442][51532] Avg episode reward: [(0, '50.980'), (1, '49.680')] -[2023-10-15 16:54:44,334][52866] Updated weights for policy 1, policy_version 53190 (0.0007) -[2023-10-15 16:54:44,700][52866] Updated weights for policy 1, policy_version 53200 (0.0007) -[2023-10-15 16:54:45,060][52866] Updated weights for policy 1, policy_version 53210 (0.0008) -[2023-10-15 16:54:46,554][52833] Updated weights for policy 0, policy_version 53030 (0.0008) -[2023-10-15 16:54:46,913][52833] Updated weights for policy 0, policy_version 53040 (0.0008) -[2023-10-15 16:54:47,286][52833] Updated weights for policy 0, policy_version 53050 (0.0008) -[2023-10-15 16:54:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 108822528. Throughput: 0: 1802.9, 1: 1800.2. Samples: 27209202. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 16:54:48,442][51532] Avg episode reward: [(0, '54.320'), (1, '50.740')] -[2023-10-15 16:54:48,724][52866] Updated weights for policy 1, policy_version 53220 (0.0008) -[2023-10-15 16:54:49,087][52866] Updated weights for policy 1, policy_version 53230 (0.0008) -[2023-10-15 16:54:49,455][52866] Updated weights for policy 1, policy_version 53240 (0.0007) -[2023-10-15 16:54:51,043][52833] Updated weights for policy 0, policy_version 53060 (0.0008) -[2023-10-15 16:54:51,418][52833] Updated weights for policy 0, policy_version 53070 (0.0008) -[2023-10-15 16:54:51,780][52833] Updated weights for policy 0, policy_version 53080 (0.0008) -[2023-10-15 16:54:53,251][52866] Updated weights for policy 1, policy_version 53250 (0.0007) -[2023-10-15 16:54:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108888064. Throughput: 0: 1812.1, 1: 1803.5. Samples: 27230690. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 16:54:53,441][51532] Avg episode reward: [(0, '50.200'), (1, '51.220')] -[2023-10-15 16:54:53,617][52866] Updated weights for policy 1, policy_version 53260 (0.0010) -[2023-10-15 16:54:53,987][52866] Updated weights for policy 1, policy_version 53270 (0.0009) -[2023-10-15 16:54:54,359][52866] Updated weights for policy 1, policy_version 53280 (0.0009) -[2023-10-15 16:54:55,484][52833] Updated weights for policy 0, policy_version 53090 (0.0009) -[2023-10-15 16:54:55,849][52833] Updated weights for policy 0, policy_version 53100 (0.0009) -[2023-10-15 16:54:56,222][52833] Updated weights for policy 0, policy_version 53110 (0.0009) -[2023-10-15 16:54:56,583][52833] Updated weights for policy 0, policy_version 53120 (0.0008) -[2023-10-15 16:54:58,221][52866] Updated weights for policy 1, policy_version 53290 (0.0008) -[2023-10-15 16:54:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 108953600. Throughput: 0: 1804.9, 1: 1809.1. Samples: 27252648. Policy #0 lag: (min: 22.0, avg: 26.5, max: 54.0) -[2023-10-15 16:54:58,442][51532] Avg episode reward: [(0, '48.010'), (1, '50.380')] -[2023-10-15 16:54:58,596][52866] Updated weights for policy 1, policy_version 53300 (0.0009) -[2023-10-15 16:54:58,960][52866] Updated weights for policy 1, policy_version 53310 (0.0007) -[2023-10-15 16:55:00,330][52833] Updated weights for policy 0, policy_version 53130 (0.0007) -[2023-10-15 16:55:00,703][52833] Updated weights for policy 0, policy_version 53140 (0.0009) -[2023-10-15 16:55:01,072][52833] Updated weights for policy 0, policy_version 53150 (0.0007) -[2023-10-15 16:55:02,722][52866] Updated weights for policy 1, policy_version 53320 (0.0008) -[2023-10-15 16:55:03,096][52866] Updated weights for policy 1, policy_version 53330 (0.0008) -[2023-10-15 16:55:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 109019136. Throughput: 0: 1816.1, 1: 1800.7. Samples: 27263196. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 16:55:03,442][51532] Avg episode reward: [(0, '46.780'), (1, '49.640')] -[2023-10-15 16:55:03,471][52866] Updated weights for policy 1, policy_version 53340 (0.0008) -[2023-10-15 16:55:04,634][52833] Updated weights for policy 0, policy_version 53160 (0.0009) -[2023-10-15 16:55:05,011][52833] Updated weights for policy 0, policy_version 53170 (0.0008) -[2023-10-15 16:55:05,379][52833] Updated weights for policy 0, policy_version 53180 (0.0007) -[2023-10-15 16:55:07,215][52866] Updated weights for policy 1, policy_version 53350 (0.0008) -[2023-10-15 16:55:07,581][52866] Updated weights for policy 1, policy_version 53360 (0.0008) -[2023-10-15 16:55:07,948][52866] Updated weights for policy 1, policy_version 53370 (0.0009) -[2023-10-15 16:55:08,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 109117440. Throughput: 0: 1808.9, 1: 1811.2. Samples: 27285424. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 16:55:08,441][51532] Avg episode reward: [(0, '48.730'), (1, '47.490')] -[2023-10-15 16:55:09,083][52833] Updated weights for policy 0, policy_version 53190 (0.0007) -[2023-10-15 16:55:09,457][52833] Updated weights for policy 0, policy_version 53200 (0.0008) -[2023-10-15 16:55:09,827][52833] Updated weights for policy 0, policy_version 53210 (0.0010) -[2023-10-15 16:55:11,682][52866] Updated weights for policy 1, policy_version 53380 (0.0008) -[2023-10-15 16:55:12,055][52866] Updated weights for policy 1, policy_version 53390 (0.0008) -[2023-10-15 16:55:12,415][52866] Updated weights for policy 1, policy_version 53400 (0.0009) -[2023-10-15 16:55:13,429][52833] Updated weights for policy 0, policy_version 53220 (0.0009) -[2023-10-15 16:55:13,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 109182976. Throughput: 0: 1804.9, 1: 1804.1. Samples: 27306580. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 16:55:13,441][51532] Avg episode reward: [(0, '48.890'), (1, '45.920')] -[2023-10-15 16:55:13,799][52833] Updated weights for policy 0, policy_version 53230 (0.0008) -[2023-10-15 16:55:14,174][52833] Updated weights for policy 0, policy_version 53240 (0.0008) -[2023-10-15 16:55:16,281][52866] Updated weights for policy 1, policy_version 53410 (0.0009) -[2023-10-15 16:55:16,686][52866] Updated weights for policy 1, policy_version 53420 (0.0010) -[2023-10-15 16:55:17,053][52866] Updated weights for policy 1, policy_version 53430 (0.0008) -[2023-10-15 16:55:17,417][52866] Updated weights for policy 1, policy_version 53440 (0.0009) -[2023-10-15 16:55:17,913][52833] Updated weights for policy 0, policy_version 53250 (0.0009) -[2023-10-15 16:55:18,277][52833] Updated weights for policy 0, policy_version 53260 (0.0007) -[2023-10-15 16:55:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 109248512. Throughput: 0: 1804.8, 1: 1809.5. Samples: 27317774. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 16:55:18,441][51532] Avg episode reward: [(0, '49.850'), (1, '46.060')] -[2023-10-15 16:55:18,649][52833] Updated weights for policy 0, policy_version 53270 (0.0008) -[2023-10-15 16:55:19,013][52833] Updated weights for policy 0, policy_version 53280 (0.0009) -[2023-10-15 16:55:21,044][52866] Updated weights for policy 1, policy_version 53450 (0.0008) -[2023-10-15 16:55:21,419][52866] Updated weights for policy 1, policy_version 53460 (0.0007) -[2023-10-15 16:55:21,784][52866] Updated weights for policy 1, policy_version 53470 (0.0007) -[2023-10-15 16:55:22,790][52833] Updated weights for policy 0, policy_version 53290 (0.0007) -[2023-10-15 16:55:23,155][52833] Updated weights for policy 0, policy_version 53300 (0.0011) -[2023-10-15 16:55:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 109314048. Throughput: 0: 1798.6, 1: 1795.9. Samples: 27338554. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 16:55:23,442][51532] Avg episode reward: [(0, '51.560'), (1, '45.990')] -[2023-10-15 16:55:23,525][52833] Updated weights for policy 0, policy_version 53310 (0.0009) -[2023-10-15 16:55:25,508][52866] Updated weights for policy 1, policy_version 53480 (0.0009) -[2023-10-15 16:55:25,877][52866] Updated weights for policy 1, policy_version 53490 (0.0010) -[2023-10-15 16:55:26,255][52866] Updated weights for policy 1, policy_version 53500 (0.0009) -[2023-10-15 16:55:27,332][52833] Updated weights for policy 0, policy_version 53320 (0.0010) -[2023-10-15 16:55:27,702][52833] Updated weights for policy 0, policy_version 53330 (0.0009) -[2023-10-15 16:55:28,074][52833] Updated weights for policy 0, policy_version 53340 (0.0010) -[2023-10-15 16:55:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 109412352. Throughput: 0: 1807.7, 1: 1792.1. Samples: 27360146. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 16:55:28,442][51532] Avg episode reward: [(0, '53.720'), (1, '49.870')] -[2023-10-15 16:55:29,992][52866] Updated weights for policy 1, policy_version 53510 (0.0009) -[2023-10-15 16:55:30,370][52866] Updated weights for policy 1, policy_version 53520 (0.0011) -[2023-10-15 16:55:30,734][52866] Updated weights for policy 1, policy_version 53530 (0.0007) -[2023-10-15 16:55:31,915][52833] Updated weights for policy 0, policy_version 53350 (0.0009) -[2023-10-15 16:55:32,294][52833] Updated weights for policy 0, policy_version 53360 (0.0008) -[2023-10-15 16:55:32,668][52833] Updated weights for policy 0, policy_version 53370 (0.0010) -[2023-10-15 16:55:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 109477888. Throughput: 0: 1796.5, 1: 1793.7. Samples: 27370762. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 16:55:33,442][51532] Avg episode reward: [(0, '50.100'), (1, '50.990')] -[2023-10-15 16:55:34,567][52866] Updated weights for policy 1, policy_version 53540 (0.0010) -[2023-10-15 16:55:34,920][52866] Updated weights for policy 1, policy_version 53550 (0.0010) -[2023-10-15 16:55:35,283][52866] Updated weights for policy 1, policy_version 53560 (0.0008) -[2023-10-15 16:55:36,428][52833] Updated weights for policy 0, policy_version 53380 (0.0010) -[2023-10-15 16:55:36,797][52833] Updated weights for policy 0, policy_version 53390 (0.0011) -[2023-10-15 16:55:37,168][52833] Updated weights for policy 0, policy_version 53400 (0.0008) -[2023-10-15 16:55:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109543424. Throughput: 0: 1804.6, 1: 1786.2. Samples: 27392274. Policy #0 lag: (min: 4.0, avg: 5.5, max: 31.0) -[2023-10-15 16:55:38,441][51532] Avg episode reward: [(0, '49.980'), (1, '48.890')] -[2023-10-15 16:55:38,940][52866] Updated weights for policy 1, policy_version 53570 (0.0007) -[2023-10-15 16:55:39,298][52866] Updated weights for policy 1, policy_version 53580 (0.0009) -[2023-10-15 16:55:39,672][52866] Updated weights for policy 1, policy_version 53590 (0.0007) -[2023-10-15 16:55:40,031][52866] Updated weights for policy 1, policy_version 53600 (0.0007) -[2023-10-15 16:55:40,918][52833] Updated weights for policy 0, policy_version 53410 (0.0008) -[2023-10-15 16:55:41,305][52833] Updated weights for policy 0, policy_version 53420 (0.0009) -[2023-10-15 16:55:41,678][52833] Updated weights for policy 0, policy_version 53430 (0.0008) -[2023-10-15 16:55:42,038][52833] Updated weights for policy 0, policy_version 53440 (0.0008) -[2023-10-15 16:55:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109608960. Throughput: 0: 1788.6, 1: 1802.3. Samples: 27414238. Policy #0 lag: (min: 4.0, avg: 5.5, max: 31.0) -[2023-10-15 16:55:43,442][51532] Avg episode reward: [(0, '53.410'), (1, '49.550')] -[2023-10-15 16:55:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000053440_54722560.pth... -[2023-10-15 16:55:43,485][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000051744_52985856.pth -[2023-10-15 16:55:43,679][52866] Updated weights for policy 1, policy_version 53610 (0.0007) -[2023-10-15 16:55:44,038][52866] Updated weights for policy 1, policy_version 53620 (0.0007) -[2023-10-15 16:55:44,405][52866] Updated weights for policy 1, policy_version 53630 (0.0010) -[2023-10-15 16:55:44,480][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000053632_54919168.pth... -[2023-10-15 16:55:44,509][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000051936_53182464.pth -[2023-10-15 16:55:45,802][52833] Updated weights for policy 0, policy_version 53450 (0.0008) -[2023-10-15 16:55:46,173][52833] Updated weights for policy 0, policy_version 53460 (0.0007) -[2023-10-15 16:55:46,545][52833] Updated weights for policy 0, policy_version 53470 (0.0010) -[2023-10-15 16:55:48,227][52866] Updated weights for policy 1, policy_version 53640 (0.0008) -[2023-10-15 16:55:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109674496. Throughput: 0: 1797.8, 1: 1796.3. Samples: 27424932. Policy #0 lag: (min: 4.0, avg: 5.5, max: 31.0) -[2023-10-15 16:55:48,441][51532] Avg episode reward: [(0, '51.820'), (1, '50.490')] -[2023-10-15 16:55:48,584][52866] Updated weights for policy 1, policy_version 53650 (0.0009) -[2023-10-15 16:55:48,948][52866] Updated weights for policy 1, policy_version 53660 (0.0009) -[2023-10-15 16:55:50,317][52833] Updated weights for policy 0, policy_version 53480 (0.0009) -[2023-10-15 16:55:50,684][52833] Updated weights for policy 0, policy_version 53490 (0.0008) -[2023-10-15 16:55:51,058][52833] Updated weights for policy 0, policy_version 53500 (0.0007) -[2023-10-15 16:55:52,579][52866] Updated weights for policy 1, policy_version 53670 (0.0008) -[2023-10-15 16:55:52,945][52866] Updated weights for policy 1, policy_version 53680 (0.0011) -[2023-10-15 16:55:53,318][52866] Updated weights for policy 1, policy_version 53690 (0.0009) -[2023-10-15 16:55:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 109740032. Throughput: 0: 1781.8, 1: 1800.0. Samples: 27446608. Policy #0 lag: (min: 4.0, avg: 5.5, max: 31.0) -[2023-10-15 16:55:53,442][51532] Avg episode reward: [(0, '49.180'), (1, '53.960')] -[2023-10-15 16:55:53,527][52518] Saving new best policy, reward=53.960! -[2023-10-15 16:55:54,886][52833] Updated weights for policy 0, policy_version 53510 (0.0011) -[2023-10-15 16:55:55,252][52833] Updated weights for policy 0, policy_version 53520 (0.0009) -[2023-10-15 16:55:55,621][52833] Updated weights for policy 0, policy_version 53530 (0.0007) -[2023-10-15 16:55:56,908][52866] Updated weights for policy 1, policy_version 53700 (0.0007) -[2023-10-15 16:55:57,276][52866] Updated weights for policy 1, policy_version 53710 (0.0007) -[2023-10-15 16:55:57,635][52866] Updated weights for policy 1, policy_version 53720 (0.0008) -[2023-10-15 16:55:58,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 109838336. Throughput: 0: 1783.7, 1: 1803.1. Samples: 27467990. Policy #0 lag: (min: 4.0, avg: 5.5, max: 31.0) -[2023-10-15 16:55:58,442][51532] Avg episode reward: [(0, '50.960'), (1, '53.380')] -[2023-10-15 16:55:59,279][52833] Updated weights for policy 0, policy_version 53540 (0.0007) -[2023-10-15 16:55:59,644][52833] Updated weights for policy 0, policy_version 53550 (0.0009) -[2023-10-15 16:56:00,015][52833] Updated weights for policy 0, policy_version 53560 (0.0007) -[2023-10-15 16:56:01,351][52866] Updated weights for policy 1, policy_version 53730 (0.0007) -[2023-10-15 16:56:01,760][52866] Updated weights for policy 1, policy_version 53740 (0.0009) -[2023-10-15 16:56:02,129][52866] Updated weights for policy 1, policy_version 53750 (0.0009) -[2023-10-15 16:56:02,507][52866] Updated weights for policy 1, policy_version 53760 (0.0010) -[2023-10-15 16:56:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 109903872. Throughput: 0: 1782.2, 1: 1809.3. Samples: 27479394. Policy #0 lag: (min: 4.0, avg: 5.5, max: 31.0) -[2023-10-15 16:56:03,442][51532] Avg episode reward: [(0, '53.160'), (1, '54.020')] -[2023-10-15 16:56:03,443][52518] Saving new best policy, reward=54.020! -[2023-10-15 16:56:03,878][52833] Updated weights for policy 0, policy_version 53570 (0.0008) -[2023-10-15 16:56:04,251][52833] Updated weights for policy 0, policy_version 53580 (0.0008) -[2023-10-15 16:56:04,618][52833] Updated weights for policy 0, policy_version 53590 (0.0008) -[2023-10-15 16:56:04,991][52833] Updated weights for policy 0, policy_version 53600 (0.0009) -[2023-10-15 16:56:06,101][52866] Updated weights for policy 1, policy_version 53770 (0.0008) -[2023-10-15 16:56:06,462][52866] Updated weights for policy 1, policy_version 53780 (0.0008) -[2023-10-15 16:56:06,836][52866] Updated weights for policy 1, policy_version 53790 (0.0011) -[2023-10-15 16:56:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 109969408. Throughput: 0: 1783.5, 1: 1812.2. Samples: 27500360. Policy #0 lag: (min: 4.0, avg: 5.5, max: 31.0) -[2023-10-15 16:56:08,442][51532] Avg episode reward: [(0, '50.650'), (1, '52.080')] -[2023-10-15 16:56:08,745][52833] Updated weights for policy 0, policy_version 53610 (0.0010) -[2023-10-15 16:56:09,109][52833] Updated weights for policy 0, policy_version 53620 (0.0007) -[2023-10-15 16:56:09,481][52833] Updated weights for policy 0, policy_version 53630 (0.0008) -[2023-10-15 16:56:10,638][52866] Updated weights for policy 1, policy_version 53800 (0.0009) -[2023-10-15 16:56:11,002][52866] Updated weights for policy 1, policy_version 53810 (0.0008) -[2023-10-15 16:56:11,366][52866] Updated weights for policy 1, policy_version 53820 (0.0009) -[2023-10-15 16:56:13,289][52833] Updated weights for policy 0, policy_version 53640 (0.0011) -[2023-10-15 16:56:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 110034944. Throughput: 0: 1806.4, 1: 1808.0. Samples: 27522798. Policy #0 lag: (min: 4.0, avg: 5.5, max: 31.0) -[2023-10-15 16:56:13,441][51532] Avg episode reward: [(0, '50.690'), (1, '48.270')] -[2023-10-15 16:56:13,662][52833] Updated weights for policy 0, policy_version 53650 (0.0008) -[2023-10-15 16:56:14,021][52833] Updated weights for policy 0, policy_version 53660 (0.0010) -[2023-10-15 16:56:15,175][52866] Updated weights for policy 1, policy_version 53830 (0.0009) -[2023-10-15 16:56:15,533][52866] Updated weights for policy 1, policy_version 53840 (0.0010) -[2023-10-15 16:56:15,914][52866] Updated weights for policy 1, policy_version 53850 (0.0008) -[2023-10-15 16:56:17,750][52833] Updated weights for policy 0, policy_version 53670 (0.0010) -[2023-10-15 16:56:18,117][52833] Updated weights for policy 0, policy_version 53680 (0.0008) -[2023-10-15 16:56:18,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 110100480. Throughput: 0: 1784.1, 1: 1817.3. Samples: 27532824. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:56:18,441][51532] Avg episode reward: [(0, '51.170'), (1, '48.050')] -[2023-10-15 16:56:18,487][52833] Updated weights for policy 0, policy_version 53690 (0.0007) -[2023-10-15 16:56:19,638][52866] Updated weights for policy 1, policy_version 53860 (0.0007) -[2023-10-15 16:56:20,002][52866] Updated weights for policy 1, policy_version 53870 (0.0007) -[2023-10-15 16:56:20,370][52866] Updated weights for policy 1, policy_version 53880 (0.0008) -[2023-10-15 16:56:22,211][52833] Updated weights for policy 0, policy_version 53700 (0.0009) -[2023-10-15 16:56:22,577][52833] Updated weights for policy 0, policy_version 53710 (0.0009) -[2023-10-15 16:56:22,939][52833] Updated weights for policy 0, policy_version 53720 (0.0011) -[2023-10-15 16:56:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 110198784. Throughput: 0: 1801.6, 1: 1817.1. Samples: 27555114. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:56:23,442][51532] Avg episode reward: [(0, '52.640'), (1, '48.950')] -[2023-10-15 16:56:23,988][52866] Updated weights for policy 1, policy_version 53890 (0.0009) -[2023-10-15 16:56:24,363][52866] Updated weights for policy 1, policy_version 53900 (0.0011) -[2023-10-15 16:56:24,724][52866] Updated weights for policy 1, policy_version 53910 (0.0010) -[2023-10-15 16:56:25,095][52866] Updated weights for policy 1, policy_version 53920 (0.0008) -[2023-10-15 16:56:26,941][52833] Updated weights for policy 0, policy_version 53730 (0.0010) -[2023-10-15 16:56:27,344][52833] Updated weights for policy 0, policy_version 53740 (0.0007) -[2023-10-15 16:56:27,717][52833] Updated weights for policy 0, policy_version 53750 (0.0007) -[2023-10-15 16:56:28,093][52833] Updated weights for policy 0, policy_version 53760 (0.0010) -[2023-10-15 16:56:28,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 110264320. Throughput: 0: 1791.0, 1: 1812.7. Samples: 27576406. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:56:28,442][51532] Avg episode reward: [(0, '51.090'), (1, '45.890')] -[2023-10-15 16:56:28,904][52866] Updated weights for policy 1, policy_version 53930 (0.0008) -[2023-10-15 16:56:29,265][52866] Updated weights for policy 1, policy_version 53940 (0.0008) -[2023-10-15 16:56:29,626][52866] Updated weights for policy 1, policy_version 53950 (0.0009) -[2023-10-15 16:56:31,806][52833] Updated weights for policy 0, policy_version 53770 (0.0008) -[2023-10-15 16:56:32,180][52833] Updated weights for policy 0, policy_version 53780 (0.0007) -[2023-10-15 16:56:32,552][52833] Updated weights for policy 0, policy_version 53790 (0.0007) -[2023-10-15 16:56:33,344][52866] Updated weights for policy 1, policy_version 53960 (0.0010) -[2023-10-15 16:56:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110329856. Throughput: 0: 1794.2, 1: 1808.1. Samples: 27587034. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:56:33,441][51532] Avg episode reward: [(0, '50.790'), (1, '47.180')] -[2023-10-15 16:56:33,720][52866] Updated weights for policy 1, policy_version 53970 (0.0008) -[2023-10-15 16:56:34,092][52866] Updated weights for policy 1, policy_version 53980 (0.0009) -[2023-10-15 16:56:36,214][52833] Updated weights for policy 0, policy_version 53800 (0.0008) -[2023-10-15 16:56:36,591][52833] Updated weights for policy 0, policy_version 53810 (0.0008) -[2023-10-15 16:56:36,952][52833] Updated weights for policy 0, policy_version 53820 (0.0008) -[2023-10-15 16:56:37,787][52866] Updated weights for policy 1, policy_version 53990 (0.0008) -[2023-10-15 16:56:38,158][52866] Updated weights for policy 1, policy_version 54000 (0.0007) -[2023-10-15 16:56:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 110395392. Throughput: 0: 1792.6, 1: 1805.2. Samples: 27608510. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:56:38,442][51532] Avg episode reward: [(0, '51.260'), (1, '44.110')] -[2023-10-15 16:56:38,518][52866] Updated weights for policy 1, policy_version 54010 (0.0007) -[2023-10-15 16:56:40,647][52833] Updated weights for policy 0, policy_version 53830 (0.0007) -[2023-10-15 16:56:41,005][52833] Updated weights for policy 0, policy_version 53840 (0.0007) -[2023-10-15 16:56:41,381][52833] Updated weights for policy 0, policy_version 53850 (0.0008) -[2023-10-15 16:56:42,336][52866] Updated weights for policy 1, policy_version 54020 (0.0008) -[2023-10-15 16:56:42,707][52866] Updated weights for policy 1, policy_version 54030 (0.0007) -[2023-10-15 16:56:43,067][52866] Updated weights for policy 1, policy_version 54040 (0.0007) -[2023-10-15 16:56:43,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 110493696. Throughput: 0: 1783.5, 1: 1813.4. Samples: 27629848. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:56:43,441][51532] Avg episode reward: [(0, '49.300'), (1, '45.380')] -[2023-10-15 16:56:45,183][52833] Updated weights for policy 0, policy_version 53860 (0.0007) -[2023-10-15 16:56:45,558][52833] Updated weights for policy 0, policy_version 53870 (0.0007) -[2023-10-15 16:56:45,930][52833] Updated weights for policy 0, policy_version 53880 (0.0008) -[2023-10-15 16:56:46,828][52866] Updated weights for policy 1, policy_version 54050 (0.0010) -[2023-10-15 16:56:47,196][52866] Updated weights for policy 1, policy_version 54060 (0.0010) -[2023-10-15 16:56:47,561][52866] Updated weights for policy 1, policy_version 54070 (0.0009) -[2023-10-15 16:56:47,929][52866] Updated weights for policy 1, policy_version 54080 (0.0011) -[2023-10-15 16:56:48,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 110559232. Throughput: 0: 1797.2, 1: 1799.2. Samples: 27641234. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 16:56:48,442][51532] Avg episode reward: [(0, '49.020'), (1, '44.100')] -[2023-10-15 16:56:49,597][52833] Updated weights for policy 0, policy_version 53890 (0.0009) -[2023-10-15 16:56:49,969][52833] Updated weights for policy 0, policy_version 53900 (0.0008) -[2023-10-15 16:56:50,333][52833] Updated weights for policy 0, policy_version 53910 (0.0007) -[2023-10-15 16:56:50,706][52833] Updated weights for policy 0, policy_version 53920 (0.0008) -[2023-10-15 16:56:51,618][52866] Updated weights for policy 1, policy_version 54090 (0.0008) -[2023-10-15 16:56:51,977][52866] Updated weights for policy 1, policy_version 54100 (0.0008) -[2023-10-15 16:56:52,346][52866] Updated weights for policy 1, policy_version 54110 (0.0007) -[2023-10-15 16:56:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 110624768. Throughput: 0: 1785.6, 1: 1815.5. Samples: 27662410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:56:53,442][51532] Avg episode reward: [(0, '48.110'), (1, '46.950')] -[2023-10-15 16:56:54,299][52833] Updated weights for policy 0, policy_version 53930 (0.0010) -[2023-10-15 16:56:54,674][52833] Updated weights for policy 0, policy_version 53940 (0.0008) -[2023-10-15 16:56:55,031][52833] Updated weights for policy 0, policy_version 53950 (0.0007) -[2023-10-15 16:56:56,060][52866] Updated weights for policy 1, policy_version 54120 (0.0009) -[2023-10-15 16:56:56,433][52866] Updated weights for policy 1, policy_version 54130 (0.0010) -[2023-10-15 16:56:56,796][52866] Updated weights for policy 1, policy_version 54140 (0.0009) -[2023-10-15 16:56:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110690304. Throughput: 0: 1783.9, 1: 1806.3. Samples: 27684358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:56:58,442][51532] Avg episode reward: [(0, '49.520'), (1, '48.860')] -[2023-10-15 16:56:58,936][52833] Updated weights for policy 0, policy_version 53960 (0.0007) -[2023-10-15 16:56:59,306][52833] Updated weights for policy 0, policy_version 53970 (0.0008) -[2023-10-15 16:56:59,677][52833] Updated weights for policy 0, policy_version 53980 (0.0009) -[2023-10-15 16:57:00,478][52866] Updated weights for policy 1, policy_version 54150 (0.0008) -[2023-10-15 16:57:00,844][52866] Updated weights for policy 1, policy_version 54160 (0.0009) -[2023-10-15 16:57:01,222][52866] Updated weights for policy 1, policy_version 54170 (0.0008) -[2023-10-15 16:57:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 110755840. Throughput: 0: 1786.0, 1: 1816.8. Samples: 27694950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:03,441][51532] Avg episode reward: [(0, '49.980'), (1, '48.370')] -[2023-10-15 16:57:03,489][52833] Updated weights for policy 0, policy_version 53990 (0.0009) -[2023-10-15 16:57:03,862][52833] Updated weights for policy 0, policy_version 54000 (0.0008) -[2023-10-15 16:57:04,231][52833] Updated weights for policy 0, policy_version 54010 (0.0007) -[2023-10-15 16:57:04,913][52866] Updated weights for policy 1, policy_version 54180 (0.0008) -[2023-10-15 16:57:05,281][52866] Updated weights for policy 1, policy_version 54190 (0.0008) -[2023-10-15 16:57:05,647][52866] Updated weights for policy 1, policy_version 54200 (0.0009) -[2023-10-15 16:57:07,993][52833] Updated weights for policy 0, policy_version 54020 (0.0009) -[2023-10-15 16:57:08,364][52833] Updated weights for policy 0, policy_version 54030 (0.0008) -[2023-10-15 16:57:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 110821376. Throughput: 0: 1785.2, 1: 1809.7. Samples: 27716886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:08,441][51532] Avg episode reward: [(0, '49.750'), (1, '49.400')] -[2023-10-15 16:57:08,727][52833] Updated weights for policy 0, policy_version 54040 (0.0007) -[2023-10-15 16:57:09,359][52866] Updated weights for policy 1, policy_version 54210 (0.0008) -[2023-10-15 16:57:09,722][52866] Updated weights for policy 1, policy_version 54220 (0.0009) -[2023-10-15 16:57:10,087][52866] Updated weights for policy 1, policy_version 54230 (0.0008) -[2023-10-15 16:57:10,447][52866] Updated weights for policy 1, policy_version 54240 (0.0007) -[2023-10-15 16:57:12,326][52833] Updated weights for policy 0, policy_version 54050 (0.0008) -[2023-10-15 16:57:12,721][52833] Updated weights for policy 0, policy_version 54060 (0.0009) -[2023-10-15 16:57:13,086][52833] Updated weights for policy 0, policy_version 54070 (0.0008) -[2023-10-15 16:57:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 110886912. Throughput: 0: 1800.7, 1: 1808.1. Samples: 27738802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:13,441][51532] Avg episode reward: [(0, '51.780'), (1, '49.690')] -[2023-10-15 16:57:13,458][52833] Updated weights for policy 0, policy_version 54080 (0.0007) -[2023-10-15 16:57:14,137][52866] Updated weights for policy 1, policy_version 54250 (0.0007) -[2023-10-15 16:57:14,513][52866] Updated weights for policy 1, policy_version 54260 (0.0008) -[2023-10-15 16:57:14,868][52866] Updated weights for policy 1, policy_version 54270 (0.0008) -[2023-10-15 16:57:17,226][52833] Updated weights for policy 0, policy_version 54090 (0.0009) -[2023-10-15 16:57:17,595][52833] Updated weights for policy 0, policy_version 54100 (0.0011) -[2023-10-15 16:57:17,971][52833] Updated weights for policy 0, policy_version 54110 (0.0011) -[2023-10-15 16:57:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 110985216. Throughput: 0: 1794.4, 1: 1818.0. Samples: 27749592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:18,442][51532] Avg episode reward: [(0, '53.660'), (1, '51.150')] -[2023-10-15 16:57:18,607][52866] Updated weights for policy 1, policy_version 54280 (0.0010) -[2023-10-15 16:57:18,978][52866] Updated weights for policy 1, policy_version 54290 (0.0009) -[2023-10-15 16:57:19,347][52866] Updated weights for policy 1, policy_version 54300 (0.0008) -[2023-10-15 16:57:21,889][52833] Updated weights for policy 0, policy_version 54120 (0.0008) -[2023-10-15 16:57:22,255][52833] Updated weights for policy 0, policy_version 54130 (0.0008) -[2023-10-15 16:57:22,638][52833] Updated weights for policy 0, policy_version 54140 (0.0007) -[2023-10-15 16:57:23,111][52866] Updated weights for policy 1, policy_version 54310 (0.0009) -[2023-10-15 16:57:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 111050752. Throughput: 0: 1812.0, 1: 1813.3. Samples: 27771646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:23,441][51532] Avg episode reward: [(0, '53.820'), (1, '53.210')] -[2023-10-15 16:57:23,476][52866] Updated weights for policy 1, policy_version 54320 (0.0007) -[2023-10-15 16:57:23,848][52866] Updated weights for policy 1, policy_version 54330 (0.0007) -[2023-10-15 16:57:26,378][52833] Updated weights for policy 0, policy_version 54150 (0.0007) -[2023-10-15 16:57:26,733][52833] Updated weights for policy 0, policy_version 54160 (0.0009) -[2023-10-15 16:57:27,112][52833] Updated weights for policy 0, policy_version 54170 (0.0010) -[2023-10-15 16:57:27,461][52866] Updated weights for policy 1, policy_version 54340 (0.0011) -[2023-10-15 16:57:27,824][52866] Updated weights for policy 1, policy_version 54350 (0.0011) -[2023-10-15 16:57:28,188][52866] Updated weights for policy 1, policy_version 54360 (0.0010) -[2023-10-15 16:57:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111116288. Throughput: 0: 1792.2, 1: 1812.4. Samples: 27792058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:28,441][51532] Avg episode reward: [(0, '54.470'), (1, '51.860')] -[2023-10-15 16:57:30,691][52833] Updated weights for policy 0, policy_version 54180 (0.0008) -[2023-10-15 16:57:31,063][52833] Updated weights for policy 0, policy_version 54190 (0.0007) -[2023-10-15 16:57:31,424][52833] Updated weights for policy 0, policy_version 54200 (0.0008) -[2023-10-15 16:57:31,945][52866] Updated weights for policy 1, policy_version 54370 (0.0009) -[2023-10-15 16:57:32,360][52866] Updated weights for policy 1, policy_version 54380 (0.0009) -[2023-10-15 16:57:32,730][52866] Updated weights for policy 1, policy_version 54390 (0.0008) -[2023-10-15 16:57:33,096][52866] Updated weights for policy 1, policy_version 54400 (0.0007) -[2023-10-15 16:57:33,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 111214592. Throughput: 0: 1803.3, 1: 1811.0. Samples: 27803876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:33,442][51532] Avg episode reward: [(0, '50.600'), (1, '51.980')] -[2023-10-15 16:57:35,157][52833] Updated weights for policy 0, policy_version 54210 (0.0009) -[2023-10-15 16:57:35,524][52833] Updated weights for policy 0, policy_version 54220 (0.0008) -[2023-10-15 16:57:35,894][52833] Updated weights for policy 0, policy_version 54230 (0.0007) -[2023-10-15 16:57:36,266][52833] Updated weights for policy 0, policy_version 54240 (0.0008) -[2023-10-15 16:57:36,878][52866] Updated weights for policy 1, policy_version 54410 (0.0011) -[2023-10-15 16:57:37,243][52866] Updated weights for policy 1, policy_version 54420 (0.0010) -[2023-10-15 16:57:37,608][52866] Updated weights for policy 1, policy_version 54430 (0.0010) -[2023-10-15 16:57:38,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 111280128. Throughput: 0: 1788.8, 1: 1817.1. Samples: 27824674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:38,442][51532] Avg episode reward: [(0, '52.360'), (1, '51.870')] -[2023-10-15 16:57:39,933][52833] Updated weights for policy 0, policy_version 54250 (0.0009) -[2023-10-15 16:57:40,297][52833] Updated weights for policy 0, policy_version 54260 (0.0007) -[2023-10-15 16:57:40,674][52833] Updated weights for policy 0, policy_version 54270 (0.0008) -[2023-10-15 16:57:41,367][52866] Updated weights for policy 1, policy_version 54440 (0.0008) -[2023-10-15 16:57:41,743][52866] Updated weights for policy 1, policy_version 54450 (0.0009) -[2023-10-15 16:57:42,120][52866] Updated weights for policy 1, policy_version 54460 (0.0008) -[2023-10-15 16:57:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 111345664. Throughput: 0: 1794.8, 1: 1806.4. Samples: 27846416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:43,442][51532] Avg episode reward: [(0, '53.690'), (1, '48.380')] -[2023-10-15 16:57:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000054272_55574528.pth... -[2023-10-15 16:57:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000054464_55771136.pth... -[2023-10-15 16:57:43,493][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000052768_54034432.pth -[2023-10-15 16:57:43,494][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000052608_53870592.pth -[2023-10-15 16:57:44,436][52833] Updated weights for policy 0, policy_version 54280 (0.0007) -[2023-10-15 16:57:44,810][52833] Updated weights for policy 0, policy_version 54290 (0.0008) -[2023-10-15 16:57:45,177][52833] Updated weights for policy 0, policy_version 54300 (0.0008) -[2023-10-15 16:57:45,953][52866] Updated weights for policy 1, policy_version 54470 (0.0007) -[2023-10-15 16:57:46,323][52866] Updated weights for policy 1, policy_version 54480 (0.0007) -[2023-10-15 16:57:46,689][52866] Updated weights for policy 1, policy_version 54490 (0.0009) -[2023-10-15 16:57:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 111411200. Throughput: 0: 1797.0, 1: 1812.3. Samples: 27857368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:48,441][51532] Avg episode reward: [(0, '53.240'), (1, '50.220')] -[2023-10-15 16:57:48,884][52833] Updated weights for policy 0, policy_version 54310 (0.0008) -[2023-10-15 16:57:49,252][52833] Updated weights for policy 0, policy_version 54320 (0.0009) -[2023-10-15 16:57:49,619][52833] Updated weights for policy 0, policy_version 54330 (0.0010) -[2023-10-15 16:57:50,541][52866] Updated weights for policy 1, policy_version 54500 (0.0008) -[2023-10-15 16:57:50,903][52866] Updated weights for policy 1, policy_version 54510 (0.0010) -[2023-10-15 16:57:51,275][52866] Updated weights for policy 1, policy_version 54520 (0.0011) -[2023-10-15 16:57:53,320][52833] Updated weights for policy 0, policy_version 54340 (0.0009) -[2023-10-15 16:57:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 111476736. Throughput: 0: 1799.5, 1: 1792.5. Samples: 27878528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:53,442][51532] Avg episode reward: [(0, '51.590'), (1, '47.090')] -[2023-10-15 16:57:53,693][52833] Updated weights for policy 0, policy_version 54350 (0.0009) -[2023-10-15 16:57:54,054][52833] Updated weights for policy 0, policy_version 54360 (0.0009) -[2023-10-15 16:57:55,048][52866] Updated weights for policy 1, policy_version 54530 (0.0010) -[2023-10-15 16:57:55,424][52866] Updated weights for policy 1, policy_version 54540 (0.0008) -[2023-10-15 16:57:55,793][52866] Updated weights for policy 1, policy_version 54550 (0.0009) -[2023-10-15 16:57:56,157][52866] Updated weights for policy 1, policy_version 54560 (0.0008) -[2023-10-15 16:57:58,012][52833] Updated weights for policy 0, policy_version 54370 (0.0007) -[2023-10-15 16:57:58,423][52833] Updated weights for policy 0, policy_version 54380 (0.0007) -[2023-10-15 16:57:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 111542272. Throughput: 0: 1815.2, 1: 1787.3. Samples: 27900912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:57:58,441][51532] Avg episode reward: [(0, '50.100'), (1, '47.510')] -[2023-10-15 16:57:58,794][52833] Updated weights for policy 0, policy_version 54390 (0.0007) -[2023-10-15 16:57:59,162][52833] Updated weights for policy 0, policy_version 54400 (0.0007) -[2023-10-15 16:58:00,070][52866] Updated weights for policy 1, policy_version 54570 (0.0009) -[2023-10-15 16:58:00,438][52866] Updated weights for policy 1, policy_version 54580 (0.0010) -[2023-10-15 16:58:00,804][52866] Updated weights for policy 1, policy_version 54590 (0.0008) -[2023-10-15 16:58:02,668][52833] Updated weights for policy 0, policy_version 54410 (0.0010) -[2023-10-15 16:58:03,038][52833] Updated weights for policy 0, policy_version 54420 (0.0010) -[2023-10-15 16:58:03,401][52833] Updated weights for policy 0, policy_version 54430 (0.0011) -[2023-10-15 16:58:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 111607808. Throughput: 0: 1799.5, 1: 1779.2. Samples: 27910632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:58:03,442][51532] Avg episode reward: [(0, '48.690'), (1, '46.380')] -[2023-10-15 16:58:04,402][52866] Updated weights for policy 1, policy_version 54600 (0.0007) -[2023-10-15 16:58:04,772][52866] Updated weights for policy 1, policy_version 54610 (0.0008) -[2023-10-15 16:58:05,146][52866] Updated weights for policy 1, policy_version 54620 (0.0007) -[2023-10-15 16:58:07,091][52833] Updated weights for policy 0, policy_version 54440 (0.0008) -[2023-10-15 16:58:07,469][52833] Updated weights for policy 0, policy_version 54450 (0.0007) -[2023-10-15 16:58:07,830][52833] Updated weights for policy 0, policy_version 54460 (0.0008) -[2023-10-15 16:58:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 111706112. Throughput: 0: 1801.9, 1: 1783.6. Samples: 27932992. Policy #0 lag: (min: 25.0, avg: 37.7, max: 57.0) -[2023-10-15 16:58:08,441][51532] Avg episode reward: [(0, '51.310'), (1, '44.620')] -[2023-10-15 16:58:08,794][52866] Updated weights for policy 1, policy_version 54630 (0.0008) -[2023-10-15 16:58:09,161][52866] Updated weights for policy 1, policy_version 54640 (0.0008) -[2023-10-15 16:58:09,529][52866] Updated weights for policy 1, policy_version 54650 (0.0008) -[2023-10-15 16:58:11,619][52833] Updated weights for policy 0, policy_version 54470 (0.0008) -[2023-10-15 16:58:11,990][52833] Updated weights for policy 0, policy_version 54480 (0.0008) -[2023-10-15 16:58:12,364][52833] Updated weights for policy 0, policy_version 54490 (0.0009) -[2023-10-15 16:58:13,226][52866] Updated weights for policy 1, policy_version 54660 (0.0009) -[2023-10-15 16:58:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 111771648. Throughput: 0: 1796.0, 1: 1811.7. Samples: 27954406. Policy #0 lag: (min: 25.0, avg: 37.7, max: 57.0) -[2023-10-15 16:58:13,442][51532] Avg episode reward: [(0, '50.290'), (1, '47.550')] -[2023-10-15 16:58:13,590][52866] Updated weights for policy 1, policy_version 54670 (0.0009) -[2023-10-15 16:58:13,956][52866] Updated weights for policy 1, policy_version 54680 (0.0011) -[2023-10-15 16:58:16,000][52833] Updated weights for policy 0, policy_version 54500 (0.0009) -[2023-10-15 16:58:16,361][52833] Updated weights for policy 0, policy_version 54510 (0.0007) -[2023-10-15 16:58:16,730][52833] Updated weights for policy 0, policy_version 54520 (0.0008) -[2023-10-15 16:58:17,691][52866] Updated weights for policy 1, policy_version 54690 (0.0011) -[2023-10-15 16:58:18,076][52866] Updated weights for policy 1, policy_version 54700 (0.0007) -[2023-10-15 16:58:18,439][52866] Updated weights for policy 1, policy_version 54710 (0.0009) -[2023-10-15 16:58:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111837184. Throughput: 0: 1807.7, 1: 1790.1. Samples: 27965776. Policy #0 lag: (min: 25.0, avg: 37.7, max: 57.0) -[2023-10-15 16:58:18,441][51532] Avg episode reward: [(0, '50.120'), (1, '47.490')] -[2023-10-15 16:58:18,799][52866] Updated weights for policy 1, policy_version 54720 (0.0008) -[2023-10-15 16:58:20,424][52833] Updated weights for policy 0, policy_version 54530 (0.0009) -[2023-10-15 16:58:20,799][52833] Updated weights for policy 0, policy_version 54540 (0.0009) -[2023-10-15 16:58:21,170][52833] Updated weights for policy 0, policy_version 54550 (0.0009) -[2023-10-15 16:58:21,537][52833] Updated weights for policy 0, policy_version 54560 (0.0009) -[2023-10-15 16:58:22,502][52866] Updated weights for policy 1, policy_version 54730 (0.0009) -[2023-10-15 16:58:22,865][52866] Updated weights for policy 1, policy_version 54740 (0.0008) -[2023-10-15 16:58:23,237][52866] Updated weights for policy 1, policy_version 54750 (0.0008) -[2023-10-15 16:58:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 111935488. Throughput: 0: 1804.1, 1: 1803.3. Samples: 27987002. Policy #0 lag: (min: 25.0, avg: 37.7, max: 57.0) -[2023-10-15 16:58:23,441][51532] Avg episode reward: [(0, '53.570'), (1, '45.310')] -[2023-10-15 16:58:25,407][52833] Updated weights for policy 0, policy_version 54570 (0.0008) -[2023-10-15 16:58:25,772][52833] Updated weights for policy 0, policy_version 54580 (0.0008) -[2023-10-15 16:58:26,147][52833] Updated weights for policy 0, policy_version 54590 (0.0008) -[2023-10-15 16:58:26,995][52866] Updated weights for policy 1, policy_version 54760 (0.0009) -[2023-10-15 16:58:27,358][52866] Updated weights for policy 1, policy_version 54770 (0.0007) -[2023-10-15 16:58:27,725][52866] Updated weights for policy 1, policy_version 54780 (0.0011) -[2023-10-15 16:58:28,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 112001024. Throughput: 0: 1799.1, 1: 1790.5. Samples: 28007946. Policy #0 lag: (min: 25.0, avg: 37.7, max: 57.0) -[2023-10-15 16:58:28,442][51532] Avg episode reward: [(0, '52.660'), (1, '46.690')] -[2023-10-15 16:58:29,894][52833] Updated weights for policy 0, policy_version 54600 (0.0008) -[2023-10-15 16:58:30,259][52833] Updated weights for policy 0, policy_version 54610 (0.0008) -[2023-10-15 16:58:30,635][52833] Updated weights for policy 0, policy_version 54620 (0.0008) -[2023-10-15 16:58:31,366][52866] Updated weights for policy 1, policy_version 54790 (0.0007) -[2023-10-15 16:58:31,737][52866] Updated weights for policy 1, policy_version 54800 (0.0009) -[2023-10-15 16:58:32,098][52866] Updated weights for policy 1, policy_version 54810 (0.0011) -[2023-10-15 16:58:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112066560. Throughput: 0: 1793.2, 1: 1803.9. Samples: 28019236. Policy #0 lag: (min: 25.0, avg: 37.7, max: 57.0) -[2023-10-15 16:58:33,441][51532] Avg episode reward: [(0, '54.610'), (1, '47.440')] -[2023-10-15 16:58:34,469][52833] Updated weights for policy 0, policy_version 54630 (0.0008) -[2023-10-15 16:58:34,835][52833] Updated weights for policy 0, policy_version 54640 (0.0009) -[2023-10-15 16:58:35,202][52833] Updated weights for policy 0, policy_version 54650 (0.0010) -[2023-10-15 16:58:35,777][52866] Updated weights for policy 1, policy_version 54820 (0.0010) -[2023-10-15 16:58:36,148][52866] Updated weights for policy 1, policy_version 54830 (0.0007) -[2023-10-15 16:58:36,519][52866] Updated weights for policy 1, policy_version 54840 (0.0007) -[2023-10-15 16:58:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112132096. Throughput: 0: 1798.5, 1: 1807.6. Samples: 28040800. Policy #0 lag: (min: 25.0, avg: 37.7, max: 57.0) -[2023-10-15 16:58:38,442][51532] Avg episode reward: [(0, '50.640'), (1, '47.400')] -[2023-10-15 16:58:38,866][52833] Updated weights for policy 0, policy_version 54660 (0.0007) -[2023-10-15 16:58:39,239][52833] Updated weights for policy 0, policy_version 54670 (0.0011) -[2023-10-15 16:58:39,606][52833] Updated weights for policy 0, policy_version 54680 (0.0009) -[2023-10-15 16:58:40,262][52866] Updated weights for policy 1, policy_version 54850 (0.0008) -[2023-10-15 16:58:40,633][52866] Updated weights for policy 1, policy_version 54860 (0.0009) -[2023-10-15 16:58:41,013][52866] Updated weights for policy 1, policy_version 54870 (0.0010) -[2023-10-15 16:58:41,377][52866] Updated weights for policy 1, policy_version 54880 (0.0011) -[2023-10-15 16:58:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112197632. Throughput: 0: 1792.3, 1: 1811.3. Samples: 28063074. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 16:58:43,441][51532] Avg episode reward: [(0, '51.190'), (1, '48.560')] -[2023-10-15 16:58:43,654][52833] Updated weights for policy 0, policy_version 54690 (0.0008) -[2023-10-15 16:58:44,070][52833] Updated weights for policy 0, policy_version 54700 (0.0011) -[2023-10-15 16:58:44,444][52833] Updated weights for policy 0, policy_version 54710 (0.0010) -[2023-10-15 16:58:44,811][52833] Updated weights for policy 0, policy_version 54720 (0.0008) -[2023-10-15 16:58:45,033][52866] Updated weights for policy 1, policy_version 54890 (0.0010) -[2023-10-15 16:58:45,399][52866] Updated weights for policy 1, policy_version 54900 (0.0009) -[2023-10-15 16:58:45,756][52866] Updated weights for policy 1, policy_version 54910 (0.0010) -[2023-10-15 16:58:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 112263168. Throughput: 0: 1786.5, 1: 1816.5. Samples: 28072768. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 16:58:48,441][51532] Avg episode reward: [(0, '50.680'), (1, '47.800')] -[2023-10-15 16:58:48,537][52833] Updated weights for policy 0, policy_version 54730 (0.0008) -[2023-10-15 16:58:48,907][52833] Updated weights for policy 0, policy_version 54740 (0.0008) -[2023-10-15 16:58:49,280][52833] Updated weights for policy 0, policy_version 54750 (0.0007) -[2023-10-15 16:58:49,543][52866] Updated weights for policy 1, policy_version 54920 (0.0007) -[2023-10-15 16:58:49,912][52866] Updated weights for policy 1, policy_version 54930 (0.0009) -[2023-10-15 16:58:50,283][52866] Updated weights for policy 1, policy_version 54940 (0.0009) -[2023-10-15 16:58:52,867][52833] Updated weights for policy 0, policy_version 54760 (0.0007) -[2023-10-15 16:58:53,238][52833] Updated weights for policy 0, policy_version 54770 (0.0010) -[2023-10-15 16:58:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 112328704. Throughput: 0: 1791.7, 1: 1812.0. Samples: 28095158. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 16:58:53,441][51532] Avg episode reward: [(0, '52.950'), (1, '48.980')] -[2023-10-15 16:58:53,613][52833] Updated weights for policy 0, policy_version 54780 (0.0008) -[2023-10-15 16:58:54,046][52866] Updated weights for policy 1, policy_version 54950 (0.0009) -[2023-10-15 16:58:54,415][52866] Updated weights for policy 1, policy_version 54960 (0.0007) -[2023-10-15 16:58:54,771][52866] Updated weights for policy 1, policy_version 54970 (0.0008) -[2023-10-15 16:58:57,195][52833] Updated weights for policy 0, policy_version 54790 (0.0008) -[2023-10-15 16:58:57,564][52833] Updated weights for policy 0, policy_version 54800 (0.0008) -[2023-10-15 16:58:57,928][52833] Updated weights for policy 0, policy_version 54810 (0.0010) -[2023-10-15 16:58:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 112427008. Throughput: 0: 1806.1, 1: 1807.3. Samples: 28117012. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 16:58:58,441][51532] Avg episode reward: [(0, '51.970'), (1, '49.300')] -[2023-10-15 16:58:58,478][52866] Updated weights for policy 1, policy_version 54980 (0.0008) -[2023-10-15 16:58:58,849][52866] Updated weights for policy 1, policy_version 54990 (0.0008) -[2023-10-15 16:58:59,212][52866] Updated weights for policy 1, policy_version 55000 (0.0009) -[2023-10-15 16:59:01,784][52833] Updated weights for policy 0, policy_version 54820 (0.0007) -[2023-10-15 16:59:02,157][52833] Updated weights for policy 0, policy_version 54830 (0.0007) -[2023-10-15 16:59:02,522][52833] Updated weights for policy 0, policy_version 54840 (0.0007) -[2023-10-15 16:59:02,906][52866] Updated weights for policy 1, policy_version 55010 (0.0008) -[2023-10-15 16:59:03,293][52866] Updated weights for policy 1, policy_version 55020 (0.0010) -[2023-10-15 16:59:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 112492544. Throughput: 0: 1792.4, 1: 1808.9. Samples: 28127834. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 16:59:03,441][51532] Avg episode reward: [(0, '50.360'), (1, '50.190')] -[2023-10-15 16:59:03,671][52866] Updated weights for policy 1, policy_version 55030 (0.0009) -[2023-10-15 16:59:04,027][52866] Updated weights for policy 1, policy_version 55040 (0.0008) -[2023-10-15 16:59:06,016][52833] Updated weights for policy 0, policy_version 54850 (0.0008) -[2023-10-15 16:59:06,381][52833] Updated weights for policy 0, policy_version 54860 (0.0011) -[2023-10-15 16:59:06,750][52833] Updated weights for policy 0, policy_version 54870 (0.0009) -[2023-10-15 16:59:07,117][52833] Updated weights for policy 0, policy_version 54880 (0.0007) -[2023-10-15 16:59:07,686][52866] Updated weights for policy 1, policy_version 55050 (0.0007) -[2023-10-15 16:59:08,052][52866] Updated weights for policy 1, policy_version 55060 (0.0007) -[2023-10-15 16:59:08,426][52866] Updated weights for policy 1, policy_version 55070 (0.0007) -[2023-10-15 16:59:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112558080. Throughput: 0: 1803.2, 1: 1810.9. Samples: 28149640. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 16:59:08,441][51532] Avg episode reward: [(0, '51.530'), (1, '50.040')] -[2023-10-15 16:59:10,890][52833] Updated weights for policy 0, policy_version 54890 (0.0009) -[2023-10-15 16:59:11,264][52833] Updated weights for policy 0, policy_version 54900 (0.0009) -[2023-10-15 16:59:11,634][52833] Updated weights for policy 0, policy_version 54910 (0.0011) -[2023-10-15 16:59:12,141][52866] Updated weights for policy 1, policy_version 55080 (0.0008) -[2023-10-15 16:59:12,516][52866] Updated weights for policy 1, policy_version 55090 (0.0009) -[2023-10-15 16:59:12,879][52866] Updated weights for policy 1, policy_version 55100 (0.0007) -[2023-10-15 16:59:13,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 112656384. Throughput: 0: 1799.5, 1: 1815.1. Samples: 28170602. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) -[2023-10-15 16:59:13,442][51532] Avg episode reward: [(0, '53.040'), (1, '52.120')] -[2023-10-15 16:59:15,294][52833] Updated weights for policy 0, policy_version 54920 (0.0011) -[2023-10-15 16:59:15,672][52833] Updated weights for policy 0, policy_version 54930 (0.0008) -[2023-10-15 16:59:16,031][52833] Updated weights for policy 0, policy_version 54940 (0.0007) -[2023-10-15 16:59:16,505][52866] Updated weights for policy 1, policy_version 55110 (0.0009) -[2023-10-15 16:59:16,869][52866] Updated weights for policy 1, policy_version 55120 (0.0007) -[2023-10-15 16:59:17,235][52866] Updated weights for policy 1, policy_version 55130 (0.0007) -[2023-10-15 16:59:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 112721920. Throughput: 0: 1817.0, 1: 1807.5. Samples: 28182336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:18,442][51532] Avg episode reward: [(0, '49.860'), (1, '49.440')] -[2023-10-15 16:59:19,899][52833] Updated weights for policy 0, policy_version 54950 (0.0009) -[2023-10-15 16:59:20,271][52833] Updated weights for policy 0, policy_version 54960 (0.0008) -[2023-10-15 16:59:20,646][52833] Updated weights for policy 0, policy_version 54970 (0.0009) -[2023-10-15 16:59:21,037][52866] Updated weights for policy 1, policy_version 55140 (0.0008) -[2023-10-15 16:59:21,410][52866] Updated weights for policy 1, policy_version 55150 (0.0008) -[2023-10-15 16:59:21,768][52866] Updated weights for policy 1, policy_version 55160 (0.0008) -[2023-10-15 16:59:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 112787456. Throughput: 0: 1797.9, 1: 1808.3. Samples: 28203078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:23,442][51532] Avg episode reward: [(0, '51.100'), (1, '47.830')] -[2023-10-15 16:59:24,418][52833] Updated weights for policy 0, policy_version 54980 (0.0009) -[2023-10-15 16:59:24,778][52833] Updated weights for policy 0, policy_version 54990 (0.0008) -[2023-10-15 16:59:25,147][52833] Updated weights for policy 0, policy_version 55000 (0.0009) -[2023-10-15 16:59:25,645][52866] Updated weights for policy 1, policy_version 55170 (0.0008) -[2023-10-15 16:59:26,016][52866] Updated weights for policy 1, policy_version 55180 (0.0011) -[2023-10-15 16:59:26,383][52866] Updated weights for policy 1, policy_version 55190 (0.0011) -[2023-10-15 16:59:26,747][52866] Updated weights for policy 1, policy_version 55200 (0.0010) -[2023-10-15 16:59:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112852992. Throughput: 0: 1806.4, 1: 1801.1. Samples: 28225414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:28,442][51532] Avg episode reward: [(0, '51.090'), (1, '44.870')] -[2023-10-15 16:59:28,703][52833] Updated weights for policy 0, policy_version 55010 (0.0008) -[2023-10-15 16:59:29,118][52833] Updated weights for policy 0, policy_version 55020 (0.0008) -[2023-10-15 16:59:29,487][52833] Updated weights for policy 0, policy_version 55030 (0.0008) -[2023-10-15 16:59:29,858][52833] Updated weights for policy 0, policy_version 55040 (0.0009) -[2023-10-15 16:59:30,515][52866] Updated weights for policy 1, policy_version 55210 (0.0009) -[2023-10-15 16:59:30,886][52866] Updated weights for policy 1, policy_version 55220 (0.0007) -[2023-10-15 16:59:31,245][52866] Updated weights for policy 1, policy_version 55230 (0.0008) -[2023-10-15 16:59:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 112918528. Throughput: 0: 1808.8, 1: 1810.7. Samples: 28235648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:33,441][51532] Avg episode reward: [(0, '48.220'), (1, '45.630')] -[2023-10-15 16:59:33,579][52833] Updated weights for policy 0, policy_version 55050 (0.0010) -[2023-10-15 16:59:33,949][52833] Updated weights for policy 0, policy_version 55060 (0.0010) -[2023-10-15 16:59:34,314][52833] Updated weights for policy 0, policy_version 55070 (0.0008) -[2023-10-15 16:59:35,059][52866] Updated weights for policy 1, policy_version 55240 (0.0008) -[2023-10-15 16:59:35,425][52866] Updated weights for policy 1, policy_version 55250 (0.0007) -[2023-10-15 16:59:35,785][52866] Updated weights for policy 1, policy_version 55260 (0.0011) -[2023-10-15 16:59:38,151][52833] Updated weights for policy 0, policy_version 55080 (0.0007) -[2023-10-15 16:59:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 112984064. Throughput: 0: 1811.7, 1: 1801.2. Samples: 28257736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:38,441][51532] Avg episode reward: [(0, '46.630'), (1, '48.170')] -[2023-10-15 16:59:38,520][52833] Updated weights for policy 0, policy_version 55090 (0.0008) -[2023-10-15 16:59:38,889][52833] Updated weights for policy 0, policy_version 55100 (0.0009) -[2023-10-15 16:59:39,528][52866] Updated weights for policy 1, policy_version 55270 (0.0011) -[2023-10-15 16:59:39,894][52866] Updated weights for policy 1, policy_version 55280 (0.0010) -[2023-10-15 16:59:40,264][52866] Updated weights for policy 1, policy_version 55290 (0.0008) -[2023-10-15 16:59:42,649][52833] Updated weights for policy 0, policy_version 55110 (0.0010) -[2023-10-15 16:59:43,029][52833] Updated weights for policy 0, policy_version 55120 (0.0007) -[2023-10-15 16:59:43,408][52833] Updated weights for policy 0, policy_version 55130 (0.0008) -[2023-10-15 16:59:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 113049600. Throughput: 0: 1813.7, 1: 1797.5. Samples: 28279514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:43,442][51532] Avg episode reward: [(0, '47.120'), (1, '48.340')] -[2023-10-15 16:59:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000055296_56623104.pth... -[2023-10-15 16:59:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000053632_54919168.pth -[2023-10-15 16:59:43,626][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000055136_56459264.pth... -[2023-10-15 16:59:43,656][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000053440_54722560.pth -[2023-10-15 16:59:44,046][52866] Updated weights for policy 1, policy_version 55300 (0.0010) -[2023-10-15 16:59:44,417][52866] Updated weights for policy 1, policy_version 55310 (0.0009) -[2023-10-15 16:59:44,785][52866] Updated weights for policy 1, policy_version 55320 (0.0008) -[2023-10-15 16:59:47,140][52833] Updated weights for policy 0, policy_version 55140 (0.0009) -[2023-10-15 16:59:47,517][52833] Updated weights for policy 0, policy_version 55150 (0.0009) -[2023-10-15 16:59:47,890][52833] Updated weights for policy 0, policy_version 55160 (0.0008) -[2023-10-15 16:59:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 113147904. Throughput: 0: 1804.6, 1: 1792.6. Samples: 28289710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:48,442][51532] Avg episode reward: [(0, '48.490'), (1, '44.090')] -[2023-10-15 16:59:48,552][52866] Updated weights for policy 1, policy_version 55330 (0.0008) -[2023-10-15 16:59:48,932][52866] Updated weights for policy 1, policy_version 55340 (0.0007) -[2023-10-15 16:59:49,298][52866] Updated weights for policy 1, policy_version 55350 (0.0009) -[2023-10-15 16:59:49,665][52866] Updated weights for policy 1, policy_version 55360 (0.0008) -[2023-10-15 16:59:51,691][52833] Updated weights for policy 0, policy_version 55170 (0.0008) -[2023-10-15 16:59:52,063][52833] Updated weights for policy 0, policy_version 55180 (0.0007) -[2023-10-15 16:59:52,427][52833] Updated weights for policy 0, policy_version 55190 (0.0007) -[2023-10-15 16:59:52,799][52833] Updated weights for policy 0, policy_version 55200 (0.0008) -[2023-10-15 16:59:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 113213440. Throughput: 0: 1813.2, 1: 1788.0. Samples: 28311696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:53,442][51532] Avg episode reward: [(0, '45.440'), (1, '44.620')] -[2023-10-15 16:59:53,452][52866] Updated weights for policy 1, policy_version 55370 (0.0010) -[2023-10-15 16:59:53,830][52866] Updated weights for policy 1, policy_version 55380 (0.0009) -[2023-10-15 16:59:54,191][52866] Updated weights for policy 1, policy_version 55390 (0.0009) -[2023-10-15 16:59:56,620][52833] Updated weights for policy 0, policy_version 55210 (0.0008) -[2023-10-15 16:59:56,983][52833] Updated weights for policy 0, policy_version 55220 (0.0009) -[2023-10-15 16:59:57,347][52833] Updated weights for policy 0, policy_version 55230 (0.0010) -[2023-10-15 16:59:57,860][52866] Updated weights for policy 1, policy_version 55400 (0.0008) -[2023-10-15 16:59:58,232][52866] Updated weights for policy 1, policy_version 55410 (0.0009) -[2023-10-15 16:59:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113278976. Throughput: 0: 1789.1, 1: 1812.8. Samples: 28332686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 16:59:58,441][51532] Avg episode reward: [(0, '46.020'), (1, '44.180')] -[2023-10-15 16:59:58,601][52866] Updated weights for policy 1, policy_version 55420 (0.0007) -[2023-10-15 17:00:01,064][52833] Updated weights for policy 0, policy_version 55240 (0.0010) -[2023-10-15 17:00:01,437][52833] Updated weights for policy 0, policy_version 55250 (0.0009) -[2023-10-15 17:00:01,803][52833] Updated weights for policy 0, policy_version 55260 (0.0008) -[2023-10-15 17:00:02,237][52866] Updated weights for policy 1, policy_version 55430 (0.0009) -[2023-10-15 17:00:02,595][52866] Updated weights for policy 1, policy_version 55440 (0.0010) -[2023-10-15 17:00:02,967][52866] Updated weights for policy 1, policy_version 55450 (0.0009) -[2023-10-15 17:00:03,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 113377280. Throughput: 0: 1807.6, 1: 1795.2. Samples: 28344464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:00:03,442][51532] Avg episode reward: [(0, '48.020'), (1, '45.540')] -[2023-10-15 17:00:05,584][52833] Updated weights for policy 0, policy_version 55270 (0.0009) -[2023-10-15 17:00:05,946][52833] Updated weights for policy 0, policy_version 55280 (0.0010) -[2023-10-15 17:00:06,322][52833] Updated weights for policy 0, policy_version 55290 (0.0009) -[2023-10-15 17:00:06,745][52866] Updated weights for policy 1, policy_version 55460 (0.0009) -[2023-10-15 17:00:07,108][52866] Updated weights for policy 1, policy_version 55470 (0.0007) -[2023-10-15 17:00:07,474][52866] Updated weights for policy 1, policy_version 55480 (0.0007) -[2023-10-15 17:00:08,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 113442816. Throughput: 0: 1792.1, 1: 1810.9. Samples: 28365214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:00:08,441][51532] Avg episode reward: [(0, '45.600'), (1, '48.760')] -[2023-10-15 17:00:09,956][52833] Updated weights for policy 0, policy_version 55300 (0.0007) -[2023-10-15 17:00:10,335][52833] Updated weights for policy 0, policy_version 55310 (0.0008) -[2023-10-15 17:00:10,695][52833] Updated weights for policy 0, policy_version 55320 (0.0011) -[2023-10-15 17:00:11,137][52866] Updated weights for policy 1, policy_version 55490 (0.0007) -[2023-10-15 17:00:11,504][52866] Updated weights for policy 1, policy_version 55500 (0.0010) -[2023-10-15 17:00:11,868][52866] Updated weights for policy 1, policy_version 55510 (0.0011) -[2023-10-15 17:00:12,234][52866] Updated weights for policy 1, policy_version 55520 (0.0009) -[2023-10-15 17:00:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113508352. Throughput: 0: 1789.4, 1: 1796.0. Samples: 28386756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:00:13,442][51532] Avg episode reward: [(0, '49.140'), (1, '48.780')] -[2023-10-15 17:00:14,635][52833] Updated weights for policy 0, policy_version 55330 (0.0008) -[2023-10-15 17:00:15,032][52833] Updated weights for policy 0, policy_version 55340 (0.0010) -[2023-10-15 17:00:15,401][52833] Updated weights for policy 0, policy_version 55350 (0.0007) -[2023-10-15 17:00:15,766][52833] Updated weights for policy 0, policy_version 55360 (0.0008) -[2023-10-15 17:00:15,886][52866] Updated weights for policy 1, policy_version 55530 (0.0007) -[2023-10-15 17:00:16,250][52866] Updated weights for policy 1, policy_version 55540 (0.0007) -[2023-10-15 17:00:16,617][52866] Updated weights for policy 1, policy_version 55550 (0.0009) -[2023-10-15 17:00:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113573888. Throughput: 0: 1789.7, 1: 1810.3. Samples: 28397648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:00:18,442][51532] Avg episode reward: [(0, '48.780'), (1, '50.410')] -[2023-10-15 17:00:19,478][52833] Updated weights for policy 0, policy_version 55370 (0.0009) -[2023-10-15 17:00:19,866][52833] Updated weights for policy 0, policy_version 55380 (0.0009) -[2023-10-15 17:00:20,226][52833] Updated weights for policy 0, policy_version 55390 (0.0010) -[2023-10-15 17:00:20,515][52866] Updated weights for policy 1, policy_version 55560 (0.0008) -[2023-10-15 17:00:20,878][52866] Updated weights for policy 1, policy_version 55570 (0.0007) -[2023-10-15 17:00:21,251][52866] Updated weights for policy 1, policy_version 55580 (0.0008) -[2023-10-15 17:00:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 113639424. Throughput: 0: 1786.0, 1: 1799.5. Samples: 28419082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:00:23,441][51532] Avg episode reward: [(0, '50.180'), (1, '51.840')] -[2023-10-15 17:00:23,944][52833] Updated weights for policy 0, policy_version 55400 (0.0009) -[2023-10-15 17:00:24,312][52833] Updated weights for policy 0, policy_version 55410 (0.0008) -[2023-10-15 17:00:24,675][52833] Updated weights for policy 0, policy_version 55420 (0.0008) -[2023-10-15 17:00:24,835][52866] Updated weights for policy 1, policy_version 55590 (0.0011) -[2023-10-15 17:00:25,206][52866] Updated weights for policy 1, policy_version 55600 (0.0008) -[2023-10-15 17:00:25,571][52866] Updated weights for policy 1, policy_version 55610 (0.0008) -[2023-10-15 17:00:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 113704960. Throughput: 0: 1798.4, 1: 1804.0. Samples: 28441618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:00:28,441][51532] Avg episode reward: [(0, '50.900'), (1, '47.800')] -[2023-10-15 17:00:28,475][52833] Updated weights for policy 0, policy_version 55430 (0.0007) -[2023-10-15 17:00:28,842][52833] Updated weights for policy 0, policy_version 55440 (0.0010) -[2023-10-15 17:00:29,217][52833] Updated weights for policy 0, policy_version 55450 (0.0010) -[2023-10-15 17:00:29,356][52866] Updated weights for policy 1, policy_version 55620 (0.0008) -[2023-10-15 17:00:29,711][52866] Updated weights for policy 1, policy_version 55630 (0.0010) -[2023-10-15 17:00:30,081][52866] Updated weights for policy 1, policy_version 55640 (0.0010) -[2023-10-15 17:00:32,883][52833] Updated weights for policy 0, policy_version 55460 (0.0009) -[2023-10-15 17:00:33,251][52833] Updated weights for policy 0, policy_version 55470 (0.0009) -[2023-10-15 17:00:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 113770496. Throughput: 0: 1782.5, 1: 1809.6. Samples: 28451354. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:00:33,442][51532] Avg episode reward: [(0, '51.190'), (1, '47.170')] -[2023-10-15 17:00:33,610][52833] Updated weights for policy 0, policy_version 55480 (0.0010) -[2023-10-15 17:00:33,723][52866] Updated weights for policy 1, policy_version 55650 (0.0009) -[2023-10-15 17:00:34,107][52866] Updated weights for policy 1, policy_version 55660 (0.0009) -[2023-10-15 17:00:34,464][52866] Updated weights for policy 1, policy_version 55670 (0.0010) -[2023-10-15 17:00:34,838][52866] Updated weights for policy 1, policy_version 55680 (0.0009) -[2023-10-15 17:00:37,531][52833] Updated weights for policy 0, policy_version 55490 (0.0010) -[2023-10-15 17:00:37,911][52833] Updated weights for policy 0, policy_version 55500 (0.0008) -[2023-10-15 17:00:38,285][52833] Updated weights for policy 0, policy_version 55510 (0.0007) -[2023-10-15 17:00:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 113836032. Throughput: 0: 1786.1, 1: 1814.7. Samples: 28473728. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:00:38,441][51532] Avg episode reward: [(0, '49.180'), (1, '48.390')] -[2023-10-15 17:00:38,534][52866] Updated weights for policy 1, policy_version 55690 (0.0007) -[2023-10-15 17:00:38,656][52833] Updated weights for policy 0, policy_version 55520 (0.0008) -[2023-10-15 17:00:38,891][52866] Updated weights for policy 1, policy_version 55700 (0.0007) -[2023-10-15 17:00:39,261][52866] Updated weights for policy 1, policy_version 55710 (0.0009) -[2023-10-15 17:00:42,592][52833] Updated weights for policy 0, policy_version 55530 (0.0010) -[2023-10-15 17:00:42,936][52866] Updated weights for policy 1, policy_version 55720 (0.0008) -[2023-10-15 17:00:42,958][52833] Updated weights for policy 0, policy_version 55540 (0.0007) -[2023-10-15 17:00:43,307][52866] Updated weights for policy 1, policy_version 55730 (0.0007) -[2023-10-15 17:00:43,327][52833] Updated weights for policy 0, policy_version 55550 (0.0007) -[2023-10-15 17:00:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 113934336. Throughput: 0: 1791.5, 1: 1811.4. Samples: 28494816. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:00:43,442][51532] Avg episode reward: [(0, '50.400'), (1, '49.540')] -[2023-10-15 17:00:43,676][52866] Updated weights for policy 1, policy_version 55740 (0.0010) -[2023-10-15 17:00:47,089][52833] Updated weights for policy 0, policy_version 55560 (0.0007) -[2023-10-15 17:00:47,432][52866] Updated weights for policy 1, policy_version 55750 (0.0008) -[2023-10-15 17:00:47,456][52833] Updated weights for policy 0, policy_version 55570 (0.0008) -[2023-10-15 17:00:47,793][52866] Updated weights for policy 1, policy_version 55760 (0.0009) -[2023-10-15 17:00:47,816][52833] Updated weights for policy 0, policy_version 55580 (0.0007) -[2023-10-15 17:00:48,147][52866] Updated weights for policy 1, policy_version 55770 (0.0010) -[2023-10-15 17:00:48,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114032640. Throughput: 0: 1773.7, 1: 1805.4. Samples: 28505524. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:00:48,442][51532] Avg episode reward: [(0, '47.940'), (1, '48.760')] -[2023-10-15 17:00:51,530][52833] Updated weights for policy 0, policy_version 55590 (0.0009) -[2023-10-15 17:00:51,900][52833] Updated weights for policy 0, policy_version 55600 (0.0009) -[2023-10-15 17:00:51,965][52866] Updated weights for policy 1, policy_version 55780 (0.0010) -[2023-10-15 17:00:52,271][52833] Updated weights for policy 0, policy_version 55610 (0.0008) -[2023-10-15 17:00:52,331][52866] Updated weights for policy 1, policy_version 55790 (0.0007) -[2023-10-15 17:00:52,703][52866] Updated weights for policy 1, policy_version 55800 (0.0009) -[2023-10-15 17:00:53,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 114098176. Throughput: 0: 1783.0, 1: 1808.7. Samples: 28526842. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:00:53,442][51532] Avg episode reward: [(0, '49.740'), (1, '46.820')] -[2023-10-15 17:00:56,205][52833] Updated weights for policy 0, policy_version 55620 (0.0009) -[2023-10-15 17:00:56,473][52866] Updated weights for policy 1, policy_version 55810 (0.0007) -[2023-10-15 17:00:56,565][52833] Updated weights for policy 0, policy_version 55630 (0.0008) -[2023-10-15 17:00:56,847][52866] Updated weights for policy 1, policy_version 55820 (0.0007) -[2023-10-15 17:00:56,935][52833] Updated weights for policy 0, policy_version 55640 (0.0008) -[2023-10-15 17:00:57,203][52866] Updated weights for policy 1, policy_version 55830 (0.0009) -[2023-10-15 17:00:57,571][52866] Updated weights for policy 1, policy_version 55840 (0.0010) -[2023-10-15 17:00:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 114163712. Throughput: 0: 1759.4, 1: 1797.8. Samples: 28546832. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:00:58,441][51532] Avg episode reward: [(0, '50.200'), (1, '47.020')] -[2023-10-15 17:01:00,747][52833] Updated weights for policy 0, policy_version 55650 (0.0007) -[2023-10-15 17:01:01,154][52833] Updated weights for policy 0, policy_version 55660 (0.0007) -[2023-10-15 17:01:01,367][52866] Updated weights for policy 1, policy_version 55850 (0.0009) -[2023-10-15 17:01:01,525][52833] Updated weights for policy 0, policy_version 55670 (0.0007) -[2023-10-15 17:01:01,729][52866] Updated weights for policy 1, policy_version 55860 (0.0008) -[2023-10-15 17:01:01,892][52833] Updated weights for policy 0, policy_version 55680 (0.0007) -[2023-10-15 17:01:02,102][52866] Updated weights for policy 1, policy_version 55870 (0.0009) -[2023-10-15 17:01:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114229248. Throughput: 0: 1787.7, 1: 1807.0. Samples: 28559410. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:01:03,441][51532] Avg episode reward: [(0, '51.080'), (1, '49.320')] -[2023-10-15 17:01:05,703][52833] Updated weights for policy 0, policy_version 55690 (0.0008) -[2023-10-15 17:01:05,752][52866] Updated weights for policy 1, policy_version 55880 (0.0009) -[2023-10-15 17:01:06,074][52833] Updated weights for policy 0, policy_version 55700 (0.0007) -[2023-10-15 17:01:06,115][52866] Updated weights for policy 1, policy_version 55890 (0.0007) -[2023-10-15 17:01:06,442][52833] Updated weights for policy 0, policy_version 55710 (0.0009) -[2023-10-15 17:01:06,485][52866] Updated weights for policy 1, policy_version 55900 (0.0009) -[2023-10-15 17:01:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114294784. Throughput: 0: 1760.4, 1: 1799.8. Samples: 28579290. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) -[2023-10-15 17:01:08,442][51532] Avg episode reward: [(0, '49.350'), (1, '49.540')] -[2023-10-15 17:01:10,084][52833] Updated weights for policy 0, policy_version 55720 (0.0008) -[2023-10-15 17:01:10,277][52866] Updated weights for policy 1, policy_version 55910 (0.0009) -[2023-10-15 17:01:10,464][52833] Updated weights for policy 0, policy_version 55730 (0.0009) -[2023-10-15 17:01:10,644][52866] Updated weights for policy 1, policy_version 55920 (0.0009) -[2023-10-15 17:01:10,833][52833] Updated weights for policy 0, policy_version 55740 (0.0008) -[2023-10-15 17:01:11,004][52866] Updated weights for policy 1, policy_version 55930 (0.0010) -[2023-10-15 17:01:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114360320. Throughput: 0: 1764.3, 1: 1795.1. Samples: 28601790. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) -[2023-10-15 17:01:13,442][51532] Avg episode reward: [(0, '49.530'), (1, '49.250')] -[2023-10-15 17:01:14,639][52833] Updated weights for policy 0, policy_version 55750 (0.0009) -[2023-10-15 17:01:14,884][52866] Updated weights for policy 1, policy_version 55940 (0.0010) -[2023-10-15 17:01:15,002][52833] Updated weights for policy 0, policy_version 55760 (0.0007) -[2023-10-15 17:01:15,259][52866] Updated weights for policy 1, policy_version 55950 (0.0007) -[2023-10-15 17:01:15,375][52833] Updated weights for policy 0, policy_version 55770 (0.0007) -[2023-10-15 17:01:15,623][52866] Updated weights for policy 1, policy_version 55960 (0.0007) -[2023-10-15 17:01:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 114425856. Throughput: 0: 1767.7, 1: 1792.1. Samples: 28611544. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) -[2023-10-15 17:01:18,442][51532] Avg episode reward: [(0, '49.260'), (1, '51.180')] -[2023-10-15 17:01:19,059][52833] Updated weights for policy 0, policy_version 55780 (0.0008) -[2023-10-15 17:01:19,415][52866] Updated weights for policy 1, policy_version 55970 (0.0010) -[2023-10-15 17:01:19,436][52833] Updated weights for policy 0, policy_version 55790 (0.0008) -[2023-10-15 17:01:19,777][52866] Updated weights for policy 1, policy_version 55980 (0.0009) -[2023-10-15 17:01:19,804][52833] Updated weights for policy 0, policy_version 55800 (0.0008) -[2023-10-15 17:01:20,156][52866] Updated weights for policy 1, policy_version 55990 (0.0008) -[2023-10-15 17:01:20,520][52866] Updated weights for policy 1, policy_version 56000 (0.0010) -[2023-10-15 17:01:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 114491392. Throughput: 0: 1775.1, 1: 1783.8. Samples: 28633876. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) -[2023-10-15 17:01:23,442][51532] Avg episode reward: [(0, '49.000'), (1, '48.990')] -[2023-10-15 17:01:23,553][52833] Updated weights for policy 0, policy_version 55810 (0.0007) -[2023-10-15 17:01:23,914][52833] Updated weights for policy 0, policy_version 55820 (0.0009) -[2023-10-15 17:01:24,285][52833] Updated weights for policy 0, policy_version 55830 (0.0008) -[2023-10-15 17:01:24,294][52866] Updated weights for policy 1, policy_version 56010 (0.0007) -[2023-10-15 17:01:24,656][52833] Updated weights for policy 0, policy_version 55840 (0.0007) -[2023-10-15 17:01:24,659][52866] Updated weights for policy 1, policy_version 56020 (0.0008) -[2023-10-15 17:01:25,027][52866] Updated weights for policy 1, policy_version 56030 (0.0009) -[2023-10-15 17:01:28,356][52833] Updated weights for policy 0, policy_version 55850 (0.0008) -[2023-10-15 17:01:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 114556928. Throughput: 0: 1797.9, 1: 1789.2. Samples: 28656234. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) -[2023-10-15 17:01:28,441][51532] Avg episode reward: [(0, '52.020'), (1, '48.410')] -[2023-10-15 17:01:28,712][52833] Updated weights for policy 0, policy_version 55860 (0.0007) -[2023-10-15 17:01:28,742][52866] Updated weights for policy 1, policy_version 56040 (0.0008) -[2023-10-15 17:01:29,080][52833] Updated weights for policy 0, policy_version 55870 (0.0007) -[2023-10-15 17:01:29,106][52866] Updated weights for policy 1, policy_version 56050 (0.0007) -[2023-10-15 17:01:29,467][52866] Updated weights for policy 1, policy_version 56060 (0.0007) -[2023-10-15 17:01:32,984][52833] Updated weights for policy 0, policy_version 55880 (0.0009) -[2023-10-15 17:01:33,359][52833] Updated weights for policy 0, policy_version 55890 (0.0009) -[2023-10-15 17:01:33,359][52866] Updated weights for policy 1, policy_version 56070 (0.0008) -[2023-10-15 17:01:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 114622464. Throughput: 0: 1781.1, 1: 1781.8. Samples: 28665854. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) -[2023-10-15 17:01:33,442][51532] Avg episode reward: [(0, '53.670'), (1, '48.460')] -[2023-10-15 17:01:33,723][52866] Updated weights for policy 1, policy_version 56080 (0.0008) -[2023-10-15 17:01:33,727][52833] Updated weights for policy 0, policy_version 55900 (0.0009) -[2023-10-15 17:01:34,086][52866] Updated weights for policy 1, policy_version 56090 (0.0009) -[2023-10-15 17:01:37,468][52833] Updated weights for policy 0, policy_version 55910 (0.0009) -[2023-10-15 17:01:37,743][52866] Updated weights for policy 1, policy_version 56100 (0.0009) -[2023-10-15 17:01:37,840][52833] Updated weights for policy 0, policy_version 55920 (0.0008) -[2023-10-15 17:01:38,107][52866] Updated weights for policy 1, policy_version 56110 (0.0007) -[2023-10-15 17:01:38,201][52833] Updated weights for policy 0, policy_version 55930 (0.0007) -[2023-10-15 17:01:38,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 114720768. Throughput: 0: 1799.8, 1: 1790.3. Samples: 28688398. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) -[2023-10-15 17:01:38,442][51532] Avg episode reward: [(0, '55.030'), (1, '48.370')] -[2023-10-15 17:01:38,468][52866] Updated weights for policy 1, policy_version 56120 (0.0009) -[2023-10-15 17:01:41,814][52833] Updated weights for policy 0, policy_version 55940 (0.0009) -[2023-10-15 17:01:42,148][52866] Updated weights for policy 1, policy_version 56130 (0.0009) -[2023-10-15 17:01:42,183][52833] Updated weights for policy 0, policy_version 55950 (0.0007) -[2023-10-15 17:01:42,514][52866] Updated weights for policy 1, policy_version 56140 (0.0009) -[2023-10-15 17:01:42,554][52833] Updated weights for policy 0, policy_version 55960 (0.0008) -[2023-10-15 17:01:42,882][52866] Updated weights for policy 1, policy_version 56150 (0.0008) -[2023-10-15 17:01:43,252][52866] Updated weights for policy 1, policy_version 56160 (0.0007) -[2023-10-15 17:01:43,441][51532] Fps is (10 sec: 19660.1, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 114819072. Throughput: 0: 1795.1, 1: 1800.6. Samples: 28708640. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:01:43,442][51532] Avg episode reward: [(0, '57.980'), (1, '46.730')] -[2023-10-15 17:01:43,456][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000056160_57507840.pth... -[2023-10-15 17:01:43,456][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000055968_57311232.pth... -[2023-10-15 17:01:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000054464_55771136.pth -[2023-10-15 17:01:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000054272_55574528.pth -[2023-10-15 17:01:46,352][52833] Updated weights for policy 0, policy_version 55970 (0.0008) -[2023-10-15 17:01:46,766][52833] Updated weights for policy 0, policy_version 55980 (0.0008) -[2023-10-15 17:01:47,062][52866] Updated weights for policy 1, policy_version 56170 (0.0007) -[2023-10-15 17:01:47,123][52833] Updated weights for policy 0, policy_version 55990 (0.0007) -[2023-10-15 17:01:47,425][52866] Updated weights for policy 1, policy_version 56180 (0.0007) -[2023-10-15 17:01:47,493][52833] Updated weights for policy 0, policy_version 56000 (0.0007) -[2023-10-15 17:01:47,789][52866] Updated weights for policy 1, policy_version 56190 (0.0009) -[2023-10-15 17:01:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114884608. Throughput: 0: 1799.3, 1: 1785.6. Samples: 28720732. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:01:48,441][51532] Avg episode reward: [(0, '53.960'), (1, '43.710')] -[2023-10-15 17:01:51,258][52833] Updated weights for policy 0, policy_version 56010 (0.0007) -[2023-10-15 17:01:51,584][52866] Updated weights for policy 1, policy_version 56200 (0.0009) -[2023-10-15 17:01:51,635][52833] Updated weights for policy 0, policy_version 56020 (0.0007) -[2023-10-15 17:01:51,948][52866] Updated weights for policy 1, policy_version 56210 (0.0007) -[2023-10-15 17:01:52,002][52833] Updated weights for policy 0, policy_version 56030 (0.0007) -[2023-10-15 17:01:52,317][52866] Updated weights for policy 1, policy_version 56220 (0.0007) -[2023-10-15 17:01:53,441][51532] Fps is (10 sec: 13108.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114950144. Throughput: 0: 1796.0, 1: 1796.0. Samples: 28740928. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:01:53,441][51532] Avg episode reward: [(0, '55.950'), (1, '42.890')] -[2023-10-15 17:01:55,691][52833] Updated weights for policy 0, policy_version 56040 (0.0007) -[2023-10-15 17:01:56,059][52833] Updated weights for policy 0, policy_version 56050 (0.0008) -[2023-10-15 17:01:56,114][52866] Updated weights for policy 1, policy_version 56230 (0.0007) -[2023-10-15 17:01:56,427][52833] Updated weights for policy 0, policy_version 56060 (0.0008) -[2023-10-15 17:01:56,485][52866] Updated weights for policy 1, policy_version 56240 (0.0008) -[2023-10-15 17:01:56,851][52866] Updated weights for policy 1, policy_version 56250 (0.0008) -[2023-10-15 17:01:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 115015680. Throughput: 0: 1791.7, 1: 1783.4. Samples: 28762672. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:01:58,442][51532] Avg episode reward: [(0, '55.280'), (1, '43.860')] -[2023-10-15 17:02:00,195][52833] Updated weights for policy 0, policy_version 56070 (0.0008) -[2023-10-15 17:02:00,571][52833] Updated weights for policy 0, policy_version 56080 (0.0008) -[2023-10-15 17:02:00,580][52866] Updated weights for policy 1, policy_version 56260 (0.0009) -[2023-10-15 17:02:00,939][52833] Updated weights for policy 0, policy_version 56090 (0.0007) -[2023-10-15 17:02:00,944][52866] Updated weights for policy 1, policy_version 56270 (0.0007) -[2023-10-15 17:02:01,319][52866] Updated weights for policy 1, policy_version 56280 (0.0008) -[2023-10-15 17:02:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115081216. Throughput: 0: 1798.4, 1: 1804.6. Samples: 28773680. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:02:03,442][51532] Avg episode reward: [(0, '56.450'), (1, '43.730')] -[2023-10-15 17:02:04,844][52833] Updated weights for policy 0, policy_version 56100 (0.0010) -[2023-10-15 17:02:04,994][52866] Updated weights for policy 1, policy_version 56290 (0.0009) -[2023-10-15 17:02:05,206][52833] Updated weights for policy 0, policy_version 56110 (0.0009) -[2023-10-15 17:02:05,361][52866] Updated weights for policy 1, policy_version 56300 (0.0007) -[2023-10-15 17:02:05,575][52833] Updated weights for policy 0, policy_version 56120 (0.0008) -[2023-10-15 17:02:05,730][52866] Updated weights for policy 1, policy_version 56310 (0.0008) -[2023-10-15 17:02:06,099][52866] Updated weights for policy 1, policy_version 56320 (0.0009) -[2023-10-15 17:02:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115146752. Throughput: 0: 1781.9, 1: 1788.5. Samples: 28794542. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:02:08,442][51532] Avg episode reward: [(0, '56.670'), (1, '45.810')] -[2023-10-15 17:02:09,421][52833] Updated weights for policy 0, policy_version 56130 (0.0007) -[2023-10-15 17:02:09,786][52833] Updated weights for policy 0, policy_version 56140 (0.0010) -[2023-10-15 17:02:09,990][52866] Updated weights for policy 1, policy_version 56330 (0.0009) -[2023-10-15 17:02:10,159][52833] Updated weights for policy 0, policy_version 56150 (0.0008) -[2023-10-15 17:02:10,356][52866] Updated weights for policy 1, policy_version 56340 (0.0009) -[2023-10-15 17:02:10,533][52833] Updated weights for policy 0, policy_version 56160 (0.0008) -[2023-10-15 17:02:10,711][52866] Updated weights for policy 1, policy_version 56350 (0.0007) -[2023-10-15 17:02:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 115212288. Throughput: 0: 1781.4, 1: 1793.6. Samples: 28817108. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:02:13,442][51532] Avg episode reward: [(0, '57.250'), (1, '46.490')] -[2023-10-15 17:02:14,294][52833] Updated weights for policy 0, policy_version 56170 (0.0008) -[2023-10-15 17:02:14,383][52866] Updated weights for policy 1, policy_version 56360 (0.0009) -[2023-10-15 17:02:14,673][52833] Updated weights for policy 0, policy_version 56180 (0.0007) -[2023-10-15 17:02:14,756][52866] Updated weights for policy 1, policy_version 56370 (0.0008) -[2023-10-15 17:02:15,039][52833] Updated weights for policy 0, policy_version 56190 (0.0007) -[2023-10-15 17:02:15,117][52866] Updated weights for policy 1, policy_version 56380 (0.0008) -[2023-10-15 17:02:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 115277824. Throughput: 0: 1784.1, 1: 1793.8. Samples: 28826860. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-15 17:02:18,442][51532] Avg episode reward: [(0, '58.380'), (1, '47.950')] -[2023-10-15 17:02:18,657][52833] Updated weights for policy 0, policy_version 56200 (0.0008) -[2023-10-15 17:02:18,890][52866] Updated weights for policy 1, policy_version 56390 (0.0008) -[2023-10-15 17:02:19,024][52833] Updated weights for policy 0, policy_version 56210 (0.0008) -[2023-10-15 17:02:19,258][52866] Updated weights for policy 1, policy_version 56400 (0.0007) -[2023-10-15 17:02:19,400][52833] Updated weights for policy 0, policy_version 56220 (0.0007) -[2023-10-15 17:02:19,620][52866] Updated weights for policy 1, policy_version 56410 (0.0008) -[2023-10-15 17:02:22,989][52833] Updated weights for policy 0, policy_version 56230 (0.0008) -[2023-10-15 17:02:23,347][52833] Updated weights for policy 0, policy_version 56240 (0.0008) -[2023-10-15 17:02:23,370][52866] Updated weights for policy 1, policy_version 56420 (0.0008) -[2023-10-15 17:02:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 115343360. Throughput: 0: 1787.3, 1: 1792.1. Samples: 28849472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:02:23,441][51532] Avg episode reward: [(0, '55.520'), (1, '47.710')] -[2023-10-15 17:02:23,711][52833] Updated weights for policy 0, policy_version 56250 (0.0008) -[2023-10-15 17:02:23,724][52866] Updated weights for policy 1, policy_version 56430 (0.0007) -[2023-10-15 17:02:24,087][52866] Updated weights for policy 1, policy_version 56440 (0.0007) -[2023-10-15 17:02:27,575][52833] Updated weights for policy 0, policy_version 56260 (0.0008) -[2023-10-15 17:02:27,809][52866] Updated weights for policy 1, policy_version 56450 (0.0008) -[2023-10-15 17:02:27,950][52833] Updated weights for policy 0, policy_version 56270 (0.0008) -[2023-10-15 17:02:28,173][52866] Updated weights for policy 1, policy_version 56460 (0.0007) -[2023-10-15 17:02:28,314][52833] Updated weights for policy 0, policy_version 56280 (0.0008) -[2023-10-15 17:02:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 115408896. Throughput: 0: 1806.2, 1: 1808.1. Samples: 28871284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:02:28,442][51532] Avg episode reward: [(0, '52.240'), (1, '46.560')] -[2023-10-15 17:02:28,534][52866] Updated weights for policy 1, policy_version 56470 (0.0007) -[2023-10-15 17:02:28,900][52866] Updated weights for policy 1, policy_version 56480 (0.0009) -[2023-10-15 17:02:32,149][52833] Updated weights for policy 0, policy_version 56290 (0.0007) -[2023-10-15 17:02:32,554][52833] Updated weights for policy 0, policy_version 56300 (0.0008) -[2023-10-15 17:02:32,565][52866] Updated weights for policy 1, policy_version 56490 (0.0008) -[2023-10-15 17:02:32,916][52833] Updated weights for policy 0, policy_version 56310 (0.0008) -[2023-10-15 17:02:32,928][52866] Updated weights for policy 1, policy_version 56500 (0.0007) -[2023-10-15 17:02:33,282][52833] Updated weights for policy 0, policy_version 56320 (0.0007) -[2023-10-15 17:02:33,292][52866] Updated weights for policy 1, policy_version 56510 (0.0007) -[2023-10-15 17:02:33,441][51532] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 115539968. Throughput: 0: 1785.8, 1: 1792.8. Samples: 28881772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:02:33,442][51532] Avg episode reward: [(0, '52.710'), (1, '49.290')] -[2023-10-15 17:02:37,105][52833] Updated weights for policy 0, policy_version 56330 (0.0007) -[2023-10-15 17:02:37,167][52866] Updated weights for policy 1, policy_version 56520 (0.0008) -[2023-10-15 17:02:37,484][52833] Updated weights for policy 0, policy_version 56340 (0.0008) -[2023-10-15 17:02:37,533][52866] Updated weights for policy 1, policy_version 56530 (0.0008) -[2023-10-15 17:02:37,851][52833] Updated weights for policy 0, policy_version 56350 (0.0008) -[2023-10-15 17:02:37,899][52866] Updated weights for policy 1, policy_version 56540 (0.0007) -[2023-10-15 17:02:38,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 115605504. Throughput: 0: 1805.1, 1: 1810.2. Samples: 28903618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:02:38,442][51532] Avg episode reward: [(0, '54.750'), (1, '51.830')] -[2023-10-15 17:02:41,546][52833] Updated weights for policy 0, policy_version 56360 (0.0008) -[2023-10-15 17:02:41,709][52866] Updated weights for policy 1, policy_version 56550 (0.0008) -[2023-10-15 17:02:41,905][52833] Updated weights for policy 0, policy_version 56370 (0.0007) -[2023-10-15 17:02:42,074][52866] Updated weights for policy 1, policy_version 56560 (0.0007) -[2023-10-15 17:02:42,276][52833] Updated weights for policy 0, policy_version 56380 (0.0007) -[2023-10-15 17:02:42,432][52866] Updated weights for policy 1, policy_version 56570 (0.0007) -[2023-10-15 17:02:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 115671040. Throughput: 0: 1779.5, 1: 1792.7. Samples: 28923422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:02:43,442][51532] Avg episode reward: [(0, '53.910'), (1, '53.570')] -[2023-10-15 17:02:46,058][52866] Updated weights for policy 1, policy_version 56580 (0.0008) -[2023-10-15 17:02:46,102][52833] Updated weights for policy 0, policy_version 56390 (0.0008) -[2023-10-15 17:02:46,433][52866] Updated weights for policy 1, policy_version 56590 (0.0008) -[2023-10-15 17:02:46,483][52833] Updated weights for policy 0, policy_version 56400 (0.0009) -[2023-10-15 17:02:46,804][52866] Updated weights for policy 1, policy_version 56600 (0.0008) -[2023-10-15 17:02:46,849][52833] Updated weights for policy 0, policy_version 56410 (0.0009) -[2023-10-15 17:02:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115736576. Throughput: 0: 1804.4, 1: 1803.1. Samples: 28936016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:02:48,442][51532] Avg episode reward: [(0, '54.550'), (1, '55.590')] -[2023-10-15 17:02:48,443][52518] Saving new best policy, reward=55.590! -[2023-10-15 17:02:50,499][52866] Updated weights for policy 1, policy_version 56610 (0.0008) -[2023-10-15 17:02:50,530][52833] Updated weights for policy 0, policy_version 56420 (0.0008) -[2023-10-15 17:02:50,870][52866] Updated weights for policy 1, policy_version 56620 (0.0007) -[2023-10-15 17:02:50,902][52833] Updated weights for policy 0, policy_version 56430 (0.0008) -[2023-10-15 17:02:51,231][52866] Updated weights for policy 1, policy_version 56630 (0.0008) -[2023-10-15 17:02:51,270][52833] Updated weights for policy 0, policy_version 56440 (0.0008) -[2023-10-15 17:02:51,595][52866] Updated weights for policy 1, policy_version 56640 (0.0008) -[2023-10-15 17:02:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 115802112. Throughput: 0: 1784.3, 1: 1792.1. Samples: 28955480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:02:53,441][51532] Avg episode reward: [(0, '58.830'), (1, '55.540')] -[2023-10-15 17:02:55,025][52833] Updated weights for policy 0, policy_version 56450 (0.0009) -[2023-10-15 17:02:55,396][52833] Updated weights for policy 0, policy_version 56460 (0.0009) -[2023-10-15 17:02:55,434][52866] Updated weights for policy 1, policy_version 56650 (0.0007) -[2023-10-15 17:02:55,772][52833] Updated weights for policy 0, policy_version 56470 (0.0008) -[2023-10-15 17:02:55,803][52866] Updated weights for policy 1, policy_version 56660 (0.0008) -[2023-10-15 17:02:56,135][52833] Updated weights for policy 0, policy_version 56480 (0.0009) -[2023-10-15 17:02:56,173][52866] Updated weights for policy 1, policy_version 56670 (0.0008) -[2023-10-15 17:02:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115867648. Throughput: 0: 1781.2, 1: 1786.9. Samples: 28977674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:02:58,442][51532] Avg episode reward: [(0, '57.380'), (1, '58.420')] -[2023-10-15 17:02:58,455][52518] Saving new best policy, reward=58.420! -[2023-10-15 17:02:59,888][52866] Updated weights for policy 1, policy_version 56680 (0.0007) -[2023-10-15 17:03:00,095][52833] Updated weights for policy 0, policy_version 56490 (0.0009) -[2023-10-15 17:03:00,258][52866] Updated weights for policy 1, policy_version 56690 (0.0007) -[2023-10-15 17:03:00,473][52833] Updated weights for policy 0, policy_version 56500 (0.0007) -[2023-10-15 17:03:00,628][52866] Updated weights for policy 1, policy_version 56700 (0.0008) -[2023-10-15 17:03:00,844][52833] Updated weights for policy 0, policy_version 56510 (0.0007) -[2023-10-15 17:03:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 115933184. Throughput: 0: 1780.2, 1: 1790.8. Samples: 28987556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:03:03,442][51532] Avg episode reward: [(0, '57.030'), (1, '56.680')] -[2023-10-15 17:03:04,440][52866] Updated weights for policy 1, policy_version 56710 (0.0008) -[2023-10-15 17:03:04,527][52833] Updated weights for policy 0, policy_version 56520 (0.0007) -[2023-10-15 17:03:04,810][52866] Updated weights for policy 1, policy_version 56720 (0.0008) -[2023-10-15 17:03:04,897][52833] Updated weights for policy 0, policy_version 56530 (0.0008) -[2023-10-15 17:03:05,164][52866] Updated weights for policy 1, policy_version 56730 (0.0007) -[2023-10-15 17:03:05,268][52833] Updated weights for policy 0, policy_version 56540 (0.0008) -[2023-10-15 17:03:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 115998720. Throughput: 0: 1776.0, 1: 1790.0. Samples: 29009942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:03:08,441][51532] Avg episode reward: [(0, '57.400'), (1, '55.800')] -[2023-10-15 17:03:08,964][52866] Updated weights for policy 1, policy_version 56740 (0.0008) -[2023-10-15 17:03:09,063][52833] Updated weights for policy 0, policy_version 56550 (0.0007) -[2023-10-15 17:03:09,318][52866] Updated weights for policy 1, policy_version 56750 (0.0009) -[2023-10-15 17:03:09,423][52833] Updated weights for policy 0, policy_version 56560 (0.0008) -[2023-10-15 17:03:09,685][52866] Updated weights for policy 1, policy_version 56760 (0.0008) -[2023-10-15 17:03:09,791][52833] Updated weights for policy 0, policy_version 56570 (0.0008) -[2023-10-15 17:03:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 116064256. Throughput: 0: 1789.3, 1: 1798.3. Samples: 29032728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:03:13,442][51532] Avg episode reward: [(0, '60.120'), (1, '57.090')] -[2023-10-15 17:03:13,483][52833] Updated weights for policy 0, policy_version 56580 (0.0008) -[2023-10-15 17:03:13,483][52866] Updated weights for policy 1, policy_version 56770 (0.0008) -[2023-10-15 17:03:13,852][52833] Updated weights for policy 0, policy_version 56590 (0.0010) -[2023-10-15 17:03:13,855][52866] Updated weights for policy 1, policy_version 56780 (0.0008) -[2023-10-15 17:03:14,222][52866] Updated weights for policy 1, policy_version 56790 (0.0008) -[2023-10-15 17:03:14,224][52833] Updated weights for policy 0, policy_version 56600 (0.0008) -[2023-10-15 17:03:14,506][52410] Saving new best policy, reward=60.120! -[2023-10-15 17:03:14,585][52866] Updated weights for policy 1, policy_version 56800 (0.0007) -[2023-10-15 17:03:18,111][52833] Updated weights for policy 0, policy_version 56610 (0.0008) -[2023-10-15 17:03:18,419][52866] Updated weights for policy 1, policy_version 56810 (0.0008) -[2023-10-15 17:03:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 116129792. Throughput: 0: 1777.0, 1: 1790.6. Samples: 29042314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:03:18,442][51532] Avg episode reward: [(0, '61.500'), (1, '60.160')] -[2023-10-15 17:03:18,483][52833] Updated weights for policy 0, policy_version 56620 (0.0008) -[2023-10-15 17:03:18,778][52866] Updated weights for policy 1, policy_version 56820 (0.0007) -[2023-10-15 17:03:18,850][52833] Updated weights for policy 0, policy_version 56630 (0.0008) -[2023-10-15 17:03:19,153][52866] Updated weights for policy 1, policy_version 56830 (0.0008) -[2023-10-15 17:03:19,211][52410] Saving new best policy, reward=61.500! -[2023-10-15 17:03:19,214][52833] Updated weights for policy 0, policy_version 56640 (0.0009) -[2023-10-15 17:03:19,219][52518] Saving new best policy, reward=60.160! -[2023-10-15 17:03:22,851][52866] Updated weights for policy 1, policy_version 56840 (0.0007) -[2023-10-15 17:03:23,016][52833] Updated weights for policy 0, policy_version 56650 (0.0009) -[2023-10-15 17:03:23,225][52866] Updated weights for policy 1, policy_version 56850 (0.0008) -[2023-10-15 17:03:23,379][52833] Updated weights for policy 0, policy_version 56660 (0.0008) -[2023-10-15 17:03:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 116195328. Throughput: 0: 1782.5, 1: 1795.7. Samples: 29064636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:03:23,441][51532] Avg episode reward: [(0, '60.830'), (1, '60.930')] -[2023-10-15 17:03:23,591][52866] Updated weights for policy 1, policy_version 56860 (0.0008) -[2023-10-15 17:03:23,735][52518] Saving new best policy, reward=60.930! -[2023-10-15 17:03:23,748][52833] Updated weights for policy 0, policy_version 56670 (0.0007) -[2023-10-15 17:03:27,341][52866] Updated weights for policy 1, policy_version 56870 (0.0008) -[2023-10-15 17:03:27,491][52833] Updated weights for policy 0, policy_version 56680 (0.0008) -[2023-10-15 17:03:27,699][52866] Updated weights for policy 1, policy_version 56880 (0.0007) -[2023-10-15 17:03:27,868][52833] Updated weights for policy 0, policy_version 56690 (0.0009) -[2023-10-15 17:03:28,065][52866] Updated weights for policy 1, policy_version 56890 (0.0008) -[2023-10-15 17:03:28,225][52833] Updated weights for policy 0, policy_version 56700 (0.0008) -[2023-10-15 17:03:28,441][51532] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 116326400. Throughput: 0: 1791.1, 1: 1802.6. Samples: 29085140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:03:28,441][51532] Avg episode reward: [(0, '60.310'), (1, '59.330')] -[2023-10-15 17:03:31,982][52833] Updated weights for policy 0, policy_version 56710 (0.0009) -[2023-10-15 17:03:32,049][52866] Updated weights for policy 1, policy_version 56900 (0.0008) -[2023-10-15 17:03:32,347][52833] Updated weights for policy 0, policy_version 56720 (0.0009) -[2023-10-15 17:03:32,410][52866] Updated weights for policy 1, policy_version 56910 (0.0008) -[2023-10-15 17:03:32,727][52833] Updated weights for policy 0, policy_version 56730 (0.0008) -[2023-10-15 17:03:32,779][52866] Updated weights for policy 1, policy_version 56920 (0.0008) -[2023-10-15 17:03:33,441][51532] Fps is (10 sec: 19660.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 116391936. Throughput: 0: 1778.7, 1: 1790.3. Samples: 29096624. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:03:33,443][51532] Avg episode reward: [(0, '66.030'), (1, '58.700')] -[2023-10-15 17:03:33,444][52410] Saving new best policy, reward=66.030! -[2023-10-15 17:03:36,482][52833] Updated weights for policy 0, policy_version 56740 (0.0008) -[2023-10-15 17:03:36,607][52866] Updated weights for policy 1, policy_version 56930 (0.0007) -[2023-10-15 17:03:36,859][52833] Updated weights for policy 0, policy_version 56750 (0.0008) -[2023-10-15 17:03:36,981][52866] Updated weights for policy 1, policy_version 56940 (0.0009) -[2023-10-15 17:03:37,221][52833] Updated weights for policy 0, policy_version 56760 (0.0007) -[2023-10-15 17:03:37,336][52866] Updated weights for policy 1, policy_version 56950 (0.0007) -[2023-10-15 17:03:37,707][52866] Updated weights for policy 1, policy_version 56960 (0.0009) -[2023-10-15 17:03:38,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116457472. Throughput: 0: 1801.0, 1: 1805.6. Samples: 29117778. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:03:38,442][51532] Avg episode reward: [(0, '62.720'), (1, '58.120')] -[2023-10-15 17:03:40,928][52833] Updated weights for policy 0, policy_version 56770 (0.0009) -[2023-10-15 17:03:41,290][52833] Updated weights for policy 0, policy_version 56780 (0.0008) -[2023-10-15 17:03:41,545][52866] Updated weights for policy 1, policy_version 56970 (0.0008) -[2023-10-15 17:03:41,660][52833] Updated weights for policy 0, policy_version 56790 (0.0009) -[2023-10-15 17:03:41,907][52866] Updated weights for policy 1, policy_version 56980 (0.0007) -[2023-10-15 17:03:42,025][52833] Updated weights for policy 0, policy_version 56800 (0.0008) -[2023-10-15 17:03:42,276][52866] Updated weights for policy 1, policy_version 56990 (0.0010) -[2023-10-15 17:03:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116523008. Throughput: 0: 1789.5, 1: 1784.6. Samples: 29138510. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:03:43,442][51532] Avg episode reward: [(0, '59.540'), (1, '56.150')] -[2023-10-15 17:03:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000056992_58359808.pth... -[2023-10-15 17:03:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000056800_58163200.pth... -[2023-10-15 17:03:43,483][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000055296_56623104.pth -[2023-10-15 17:03:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000055136_56459264.pth -[2023-10-15 17:03:45,588][52833] Updated weights for policy 0, policy_version 56810 (0.0009) -[2023-10-15 17:03:45,871][52866] Updated weights for policy 1, policy_version 57000 (0.0009) -[2023-10-15 17:03:45,953][52833] Updated weights for policy 0, policy_version 56820 (0.0007) -[2023-10-15 17:03:46,241][52866] Updated weights for policy 1, policy_version 57010 (0.0009) -[2023-10-15 17:03:46,322][52833] Updated weights for policy 0, policy_version 56830 (0.0008) -[2023-10-15 17:03:46,596][52866] Updated weights for policy 1, policy_version 57020 (0.0009) -[2023-10-15 17:03:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116588544. Throughput: 0: 1806.5, 1: 1806.0. Samples: 29150118. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:03:48,441][51532] Avg episode reward: [(0, '57.900'), (1, '56.110')] -[2023-10-15 17:03:50,176][52833] Updated weights for policy 0, policy_version 56840 (0.0008) -[2023-10-15 17:03:50,417][52866] Updated weights for policy 1, policy_version 57030 (0.0009) -[2023-10-15 17:03:50,533][52833] Updated weights for policy 0, policy_version 56850 (0.0008) -[2023-10-15 17:03:50,784][52866] Updated weights for policy 1, policy_version 57040 (0.0008) -[2023-10-15 17:03:50,902][52833] Updated weights for policy 0, policy_version 56860 (0.0007) -[2023-10-15 17:03:51,147][52866] Updated weights for policy 1, policy_version 57050 (0.0007) -[2023-10-15 17:03:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 116654080. Throughput: 0: 1786.7, 1: 1782.8. Samples: 29170568. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:03:53,442][51532] Avg episode reward: [(0, '56.120'), (1, '54.250')] -[2023-10-15 17:03:54,692][52833] Updated weights for policy 0, policy_version 56870 (0.0008) -[2023-10-15 17:03:54,981][52866] Updated weights for policy 1, policy_version 57060 (0.0009) -[2023-10-15 17:03:55,069][52833] Updated weights for policy 0, policy_version 56880 (0.0010) -[2023-10-15 17:03:55,358][52866] Updated weights for policy 1, policy_version 57070 (0.0009) -[2023-10-15 17:03:55,428][52833] Updated weights for policy 0, policy_version 56890 (0.0009) -[2023-10-15 17:03:55,724][52866] Updated weights for policy 1, policy_version 57080 (0.0008) -[2023-10-15 17:03:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 116719616. Throughput: 0: 1784.8, 1: 1782.1. Samples: 29193242. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:03:58,442][51532] Avg episode reward: [(0, '53.420'), (1, '54.430')] -[2023-10-15 17:03:59,182][52833] Updated weights for policy 0, policy_version 56900 (0.0008) -[2023-10-15 17:03:59,248][52866] Updated weights for policy 1, policy_version 57090 (0.0008) -[2023-10-15 17:03:59,548][52833] Updated weights for policy 0, policy_version 56910 (0.0009) -[2023-10-15 17:03:59,612][52866] Updated weights for policy 1, policy_version 57100 (0.0008) -[2023-10-15 17:03:59,922][52833] Updated weights for policy 0, policy_version 56920 (0.0008) -[2023-10-15 17:03:59,988][52866] Updated weights for policy 1, policy_version 57110 (0.0008) -[2023-10-15 17:04:00,350][52866] Updated weights for policy 1, policy_version 57120 (0.0008) -[2023-10-15 17:04:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 116785152. Throughput: 0: 1786.2, 1: 1785.4. Samples: 29203036. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:04:03,442][51532] Avg episode reward: [(0, '50.080'), (1, '57.460')] -[2023-10-15 17:04:03,808][52833] Updated weights for policy 0, policy_version 56930 (0.0008) -[2023-10-15 17:04:04,177][52866] Updated weights for policy 1, policy_version 57130 (0.0009) -[2023-10-15 17:04:04,213][52833] Updated weights for policy 0, policy_version 56940 (0.0009) -[2023-10-15 17:04:04,537][52866] Updated weights for policy 1, policy_version 57140 (0.0008) -[2023-10-15 17:04:04,592][52833] Updated weights for policy 0, policy_version 56950 (0.0008) -[2023-10-15 17:04:04,902][52866] Updated weights for policy 1, policy_version 57150 (0.0007) -[2023-10-15 17:04:04,954][52833] Updated weights for policy 0, policy_version 56960 (0.0008) -[2023-10-15 17:04:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 116850688. Throughput: 0: 1786.8, 1: 1783.1. Samples: 29225284. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:04:08,441][51532] Avg episode reward: [(0, '52.430'), (1, '55.030')] -[2023-10-15 17:04:08,589][52866] Updated weights for policy 1, policy_version 57160 (0.0008) -[2023-10-15 17:04:08,668][52833] Updated weights for policy 0, policy_version 56970 (0.0008) -[2023-10-15 17:04:08,964][52866] Updated weights for policy 1, policy_version 57170 (0.0008) -[2023-10-15 17:04:09,043][52833] Updated weights for policy 0, policy_version 56980 (0.0007) -[2023-10-15 17:04:09,333][52866] Updated weights for policy 1, policy_version 57180 (0.0008) -[2023-10-15 17:04:09,405][52833] Updated weights for policy 0, policy_version 56990 (0.0007) -[2023-10-15 17:04:13,140][52866] Updated weights for policy 1, policy_version 57190 (0.0009) -[2023-10-15 17:04:13,156][52833] Updated weights for policy 0, policy_version 57000 (0.0008) -[2023-10-15 17:04:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 116916224. Throughput: 0: 1805.0, 1: 1800.6. Samples: 29247390. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 17:04:13,442][51532] Avg episode reward: [(0, '56.890'), (1, '53.190')] -[2023-10-15 17:04:13,505][52866] Updated weights for policy 1, policy_version 57200 (0.0007) -[2023-10-15 17:04:13,526][52833] Updated weights for policy 0, policy_version 57010 (0.0009) -[2023-10-15 17:04:13,866][52866] Updated weights for policy 1, policy_version 57210 (0.0009) -[2023-10-15 17:04:13,908][52833] Updated weights for policy 0, policy_version 57020 (0.0009) -[2023-10-15 17:04:17,671][52833] Updated weights for policy 0, policy_version 57030 (0.0008) -[2023-10-15 17:04:17,688][52866] Updated weights for policy 1, policy_version 57220 (0.0007) -[2023-10-15 17:04:18,039][52833] Updated weights for policy 0, policy_version 57040 (0.0008) -[2023-10-15 17:04:18,061][52866] Updated weights for policy 1, policy_version 57230 (0.0007) -[2023-10-15 17:04:18,403][52833] Updated weights for policy 0, policy_version 57050 (0.0008) -[2023-10-15 17:04:18,430][52866] Updated weights for policy 1, policy_version 57240 (0.0008) -[2023-10-15 17:04:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 116981760. Throughput: 0: 1782.2, 1: 1781.7. Samples: 29257002. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 17:04:18,441][51532] Avg episode reward: [(0, '58.050'), (1, '54.120')] -[2023-10-15 17:04:22,268][52833] Updated weights for policy 0, policy_version 57060 (0.0008) -[2023-10-15 17:04:22,282][52866] Updated weights for policy 1, policy_version 57250 (0.0008) -[2023-10-15 17:04:22,627][52833] Updated weights for policy 0, policy_version 57070 (0.0007) -[2023-10-15 17:04:22,647][52866] Updated weights for policy 1, policy_version 57260 (0.0007) -[2023-10-15 17:04:22,999][52833] Updated weights for policy 0, policy_version 57080 (0.0008) -[2023-10-15 17:04:23,011][52866] Updated weights for policy 1, policy_version 57270 (0.0007) -[2023-10-15 17:04:23,376][52866] Updated weights for policy 1, policy_version 57280 (0.0009) -[2023-10-15 17:04:23,441][51532] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 117112832. Throughput: 0: 1793.3, 1: 1796.6. Samples: 29279322. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 17:04:23,442][51532] Avg episode reward: [(0, '55.440'), (1, '55.440')] -[2023-10-15 17:04:26,757][52833] Updated weights for policy 0, policy_version 57090 (0.0007) -[2023-10-15 17:04:27,124][52833] Updated weights for policy 0, policy_version 57100 (0.0007) -[2023-10-15 17:04:27,153][52866] Updated weights for policy 1, policy_version 57290 (0.0009) -[2023-10-15 17:04:27,487][52833] Updated weights for policy 0, policy_version 57110 (0.0008) -[2023-10-15 17:04:27,521][52866] Updated weights for policy 1, policy_version 57300 (0.0007) -[2023-10-15 17:04:27,853][52833] Updated weights for policy 0, policy_version 57120 (0.0007) -[2023-10-15 17:04:27,892][52866] Updated weights for policy 1, policy_version 57310 (0.0008) -[2023-10-15 17:04:28,441][51532] Fps is (10 sec: 19660.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 117178368. Throughput: 0: 1775.5, 1: 1786.7. Samples: 29298808. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 17:04:28,442][51532] Avg episode reward: [(0, '55.570'), (1, '55.170')] -[2023-10-15 17:04:31,591][52866] Updated weights for policy 1, policy_version 57320 (0.0010) -[2023-10-15 17:04:31,785][52833] Updated weights for policy 0, policy_version 57130 (0.0010) -[2023-10-15 17:04:31,961][52866] Updated weights for policy 1, policy_version 57330 (0.0008) -[2023-10-15 17:04:32,148][52833] Updated weights for policy 0, policy_version 57140 (0.0009) -[2023-10-15 17:04:32,333][52866] Updated weights for policy 1, policy_version 57340 (0.0009) -[2023-10-15 17:04:32,523][52833] Updated weights for policy 0, policy_version 57150 (0.0007) -[2023-10-15 17:04:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117243904. Throughput: 0: 1789.2, 1: 1793.0. Samples: 29311318. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 17:04:33,442][51532] Avg episode reward: [(0, '56.470'), (1, '52.440')] -[2023-10-15 17:04:36,136][52866] Updated weights for policy 1, policy_version 57350 (0.0008) -[2023-10-15 17:04:36,232][52833] Updated weights for policy 0, policy_version 57160 (0.0008) -[2023-10-15 17:04:36,499][52866] Updated weights for policy 1, policy_version 57360 (0.0008) -[2023-10-15 17:04:36,600][52833] Updated weights for policy 0, policy_version 57170 (0.0009) -[2023-10-15 17:04:36,857][52866] Updated weights for policy 1, policy_version 57370 (0.0007) -[2023-10-15 17:04:36,966][52833] Updated weights for policy 0, policy_version 57180 (0.0011) -[2023-10-15 17:04:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117309440. Throughput: 0: 1785.4, 1: 1788.2. Samples: 29331382. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 17:04:38,442][51532] Avg episode reward: [(0, '57.630'), (1, '53.400')] -[2023-10-15 17:04:40,531][52866] Updated weights for policy 1, policy_version 57380 (0.0010) -[2023-10-15 17:04:40,891][52866] Updated weights for policy 1, policy_version 57390 (0.0008) -[2023-10-15 17:04:40,983][52833] Updated weights for policy 0, policy_version 57190 (0.0009) -[2023-10-15 17:04:41,255][52866] Updated weights for policy 1, policy_version 57400 (0.0009) -[2023-10-15 17:04:41,354][52833] Updated weights for policy 0, policy_version 57200 (0.0008) -[2023-10-15 17:04:41,717][52833] Updated weights for policy 0, policy_version 57210 (0.0008) -[2023-10-15 17:04:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 117374976. Throughput: 0: 1766.7, 1: 1783.1. Samples: 29352982. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-15 17:04:43,441][51532] Avg episode reward: [(0, '60.390'), (1, '52.770')] -[2023-10-15 17:04:45,067][52866] Updated weights for policy 1, policy_version 57410 (0.0010) -[2023-10-15 17:04:45,426][52866] Updated weights for policy 1, policy_version 57420 (0.0010) -[2023-10-15 17:04:45,482][52833] Updated weights for policy 0, policy_version 57220 (0.0008) -[2023-10-15 17:04:45,788][52866] Updated weights for policy 1, policy_version 57430 (0.0009) -[2023-10-15 17:04:45,854][52833] Updated weights for policy 0, policy_version 57230 (0.0007) -[2023-10-15 17:04:46,152][52866] Updated weights for policy 1, policy_version 57440 (0.0008) -[2023-10-15 17:04:46,227][52833] Updated weights for policy 0, policy_version 57240 (0.0009) -[2023-10-15 17:04:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 117440512. Throughput: 0: 1788.6, 1: 1784.1. Samples: 29363808. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) -[2023-10-15 17:04:48,441][51532] Avg episode reward: [(0, '61.120'), (1, '52.050')] -[2023-10-15 17:04:50,123][52833] Updated weights for policy 0, policy_version 57250 (0.0009) -[2023-10-15 17:04:50,151][52866] Updated weights for policy 1, policy_version 57450 (0.0008) -[2023-10-15 17:04:50,490][52833] Updated weights for policy 0, policy_version 57260 (0.0008) -[2023-10-15 17:04:50,523][52866] Updated weights for policy 1, policy_version 57460 (0.0008) -[2023-10-15 17:04:50,851][52833] Updated weights for policy 0, policy_version 57270 (0.0008) -[2023-10-15 17:04:50,885][52866] Updated weights for policy 1, policy_version 57470 (0.0009) -[2023-10-15 17:04:51,217][52833] Updated weights for policy 0, policy_version 57280 (0.0007) -[2023-10-15 17:04:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 117506048. Throughput: 0: 1765.9, 1: 1769.9. Samples: 29384392. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) -[2023-10-15 17:04:53,442][51532] Avg episode reward: [(0, '60.710'), (1, '49.830')] -[2023-10-15 17:04:54,598][52866] Updated weights for policy 1, policy_version 57480 (0.0008) -[2023-10-15 17:04:54,932][52833] Updated weights for policy 0, policy_version 57290 (0.0009) -[2023-10-15 17:04:54,964][52866] Updated weights for policy 1, policy_version 57490 (0.0008) -[2023-10-15 17:04:55,305][52833] Updated weights for policy 0, policy_version 57300 (0.0008) -[2023-10-15 17:04:55,340][52866] Updated weights for policy 1, policy_version 57500 (0.0009) -[2023-10-15 17:04:55,665][52833] Updated weights for policy 0, policy_version 57310 (0.0010) -[2023-10-15 17:04:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 117571584. Throughput: 0: 1770.5, 1: 1778.7. Samples: 29407100. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) -[2023-10-15 17:04:58,441][51532] Avg episode reward: [(0, '58.630'), (1, '48.380')] -[2023-10-15 17:04:59,113][52866] Updated weights for policy 1, policy_version 57510 (0.0009) -[2023-10-15 17:04:59,374][52833] Updated weights for policy 0, policy_version 57320 (0.0009) -[2023-10-15 17:04:59,474][52866] Updated weights for policy 1, policy_version 57520 (0.0007) -[2023-10-15 17:04:59,744][52833] Updated weights for policy 0, policy_version 57330 (0.0007) -[2023-10-15 17:04:59,843][52866] Updated weights for policy 1, policy_version 57530 (0.0007) -[2023-10-15 17:05:00,110][52833] Updated weights for policy 0, policy_version 57340 (0.0009) -[2023-10-15 17:05:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 117637120. Throughput: 0: 1776.1, 1: 1781.5. Samples: 29417096. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) -[2023-10-15 17:05:03,442][51532] Avg episode reward: [(0, '56.030'), (1, '49.620')] -[2023-10-15 17:05:03,525][52866] Updated weights for policy 1, policy_version 57540 (0.0009) -[2023-10-15 17:05:03,828][52833] Updated weights for policy 0, policy_version 57350 (0.0010) -[2023-10-15 17:05:03,887][52866] Updated weights for policy 1, policy_version 57550 (0.0008) -[2023-10-15 17:05:04,207][52833] Updated weights for policy 0, policy_version 57360 (0.0008) -[2023-10-15 17:05:04,250][52866] Updated weights for policy 1, policy_version 57560 (0.0009) -[2023-10-15 17:05:04,583][52833] Updated weights for policy 0, policy_version 57370 (0.0007) -[2023-10-15 17:05:08,107][52866] Updated weights for policy 1, policy_version 57570 (0.0007) -[2023-10-15 17:05:08,267][52833] Updated weights for policy 0, policy_version 57380 (0.0008) -[2023-10-15 17:05:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 117702656. Throughput: 0: 1777.0, 1: 1785.3. Samples: 29439624. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) -[2023-10-15 17:05:08,441][51532] Avg episode reward: [(0, '53.500'), (1, '51.300')] -[2023-10-15 17:05:08,480][52866] Updated weights for policy 1, policy_version 57580 (0.0009) -[2023-10-15 17:05:08,626][52833] Updated weights for policy 0, policy_version 57390 (0.0009) -[2023-10-15 17:05:08,845][52866] Updated weights for policy 1, policy_version 57590 (0.0008) -[2023-10-15 17:05:09,000][52833] Updated weights for policy 0, policy_version 57400 (0.0009) -[2023-10-15 17:05:09,207][52866] Updated weights for policy 1, policy_version 57600 (0.0008) -[2023-10-15 17:05:12,695][52833] Updated weights for policy 0, policy_version 57410 (0.0008) -[2023-10-15 17:05:13,000][52866] Updated weights for policy 1, policy_version 57610 (0.0007) -[2023-10-15 17:05:13,058][52833] Updated weights for policy 0, policy_version 57420 (0.0007) -[2023-10-15 17:05:13,374][52866] Updated weights for policy 1, policy_version 57620 (0.0008) -[2023-10-15 17:05:13,428][52833] Updated weights for policy 0, policy_version 57430 (0.0008) -[2023-10-15 17:05:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 117768192. Throughput: 0: 1801.6, 1: 1802.1. Samples: 29460976. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) -[2023-10-15 17:05:13,442][51532] Avg episode reward: [(0, '53.150'), (1, '50.590')] -[2023-10-15 17:05:13,731][52866] Updated weights for policy 1, policy_version 57630 (0.0008) -[2023-10-15 17:05:13,792][52833] Updated weights for policy 0, policy_version 57440 (0.0007) -[2023-10-15 17:05:17,425][52866] Updated weights for policy 1, policy_version 57640 (0.0008) -[2023-10-15 17:05:17,613][52833] Updated weights for policy 0, policy_version 57450 (0.0008) -[2023-10-15 17:05:17,798][52866] Updated weights for policy 1, policy_version 57650 (0.0007) -[2023-10-15 17:05:17,982][52833] Updated weights for policy 0, policy_version 57460 (0.0007) -[2023-10-15 17:05:18,173][52866] Updated weights for policy 1, policy_version 57660 (0.0007) -[2023-10-15 17:05:18,354][52833] Updated weights for policy 0, policy_version 57470 (0.0008) -[2023-10-15 17:05:18,441][51532] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 117899264. Throughput: 0: 1776.3, 1: 1781.8. Samples: 29471432. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) -[2023-10-15 17:05:18,442][51532] Avg episode reward: [(0, '53.270'), (1, '49.110')] -[2023-10-15 17:05:21,776][52866] Updated weights for policy 1, policy_version 57670 (0.0009) -[2023-10-15 17:05:22,094][52833] Updated weights for policy 0, policy_version 57480 (0.0007) -[2023-10-15 17:05:22,146][52866] Updated weights for policy 1, policy_version 57680 (0.0009) -[2023-10-15 17:05:22,475][52833] Updated weights for policy 0, policy_version 57490 (0.0008) -[2023-10-15 17:05:22,518][52866] Updated weights for policy 1, policy_version 57690 (0.0009) -[2023-10-15 17:05:22,843][52833] Updated weights for policy 0, policy_version 57500 (0.0010) -[2023-10-15 17:05:23,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117964800. Throughput: 0: 1795.8, 1: 1804.8. Samples: 29493410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:05:23,442][51532] Avg episode reward: [(0, '51.860'), (1, '51.890')] -[2023-10-15 17:05:26,211][52866] Updated weights for policy 1, policy_version 57700 (0.0009) -[2023-10-15 17:05:26,569][52866] Updated weights for policy 1, policy_version 57710 (0.0010) -[2023-10-15 17:05:26,585][52833] Updated weights for policy 0, policy_version 57510 (0.0008) -[2023-10-15 17:05:26,948][52866] Updated weights for policy 1, policy_version 57720 (0.0007) -[2023-10-15 17:05:26,950][52833] Updated weights for policy 0, policy_version 57520 (0.0007) -[2023-10-15 17:05:27,318][52833] Updated weights for policy 0, policy_version 57530 (0.0007) -[2023-10-15 17:05:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118030336. Throughput: 0: 1781.1, 1: 1790.0. Samples: 29513684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:05:28,442][51532] Avg episode reward: [(0, '49.710'), (1, '52.380')] -[2023-10-15 17:05:30,694][52866] Updated weights for policy 1, policy_version 57730 (0.0008) -[2023-10-15 17:05:31,066][52866] Updated weights for policy 1, policy_version 57740 (0.0010) -[2023-10-15 17:05:31,191][52833] Updated weights for policy 0, policy_version 57540 (0.0007) -[2023-10-15 17:05:31,423][52866] Updated weights for policy 1, policy_version 57750 (0.0009) -[2023-10-15 17:05:31,557][52833] Updated weights for policy 0, policy_version 57550 (0.0009) -[2023-10-15 17:05:31,788][52866] Updated weights for policy 1, policy_version 57760 (0.0008) -[2023-10-15 17:05:31,924][52833] Updated weights for policy 0, policy_version 57560 (0.0008) -[2023-10-15 17:05:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118095872. Throughput: 0: 1788.0, 1: 1807.7. Samples: 29525616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:05:33,442][51532] Avg episode reward: [(0, '47.740'), (1, '51.270')] -[2023-10-15 17:05:35,588][52866] Updated weights for policy 1, policy_version 57770 (0.0007) -[2023-10-15 17:05:35,657][52833] Updated weights for policy 0, policy_version 57570 (0.0009) -[2023-10-15 17:05:35,949][52866] Updated weights for policy 1, policy_version 57780 (0.0008) -[2023-10-15 17:05:36,036][52833] Updated weights for policy 0, policy_version 57580 (0.0008) -[2023-10-15 17:05:36,325][52866] Updated weights for policy 1, policy_version 57790 (0.0009) -[2023-10-15 17:05:36,401][52833] Updated weights for policy 0, policy_version 57590 (0.0008) -[2023-10-15 17:05:36,766][52833] Updated weights for policy 0, policy_version 57600 (0.0009) -[2023-10-15 17:05:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 118161408. Throughput: 0: 1780.6, 1: 1798.6. Samples: 29545458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:05:38,442][51532] Avg episode reward: [(0, '47.950'), (1, '51.950')] -[2023-10-15 17:05:40,202][52866] Updated weights for policy 1, policy_version 57800 (0.0008) -[2023-10-15 17:05:40,559][52866] Updated weights for policy 1, policy_version 57810 (0.0007) -[2023-10-15 17:05:40,745][52833] Updated weights for policy 0, policy_version 57610 (0.0007) -[2023-10-15 17:05:40,924][52866] Updated weights for policy 1, policy_version 57820 (0.0007) -[2023-10-15 17:05:41,115][52833] Updated weights for policy 0, policy_version 57620 (0.0008) -[2023-10-15 17:05:41,483][52833] Updated weights for policy 0, policy_version 57630 (0.0010) -[2023-10-15 17:05:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 118226944. Throughput: 0: 1774.1, 1: 1795.7. Samples: 29567742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:05:43,442][51532] Avg episode reward: [(0, '51.430'), (1, '52.600')] -[2023-10-15 17:05:43,456][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000057824_59211776.pth... -[2023-10-15 17:05:43,456][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000057632_59015168.pth... -[2023-10-15 17:05:43,489][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000056160_57507840.pth -[2023-10-15 17:05:43,495][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000055968_57311232.pth -[2023-10-15 17:05:44,551][52866] Updated weights for policy 1, policy_version 57830 (0.0010) -[2023-10-15 17:05:44,917][52866] Updated weights for policy 1, policy_version 57840 (0.0010) -[2023-10-15 17:05:45,278][52833] Updated weights for policy 0, policy_version 57640 (0.0007) -[2023-10-15 17:05:45,291][52866] Updated weights for policy 1, policy_version 57850 (0.0008) -[2023-10-15 17:05:45,646][52833] Updated weights for policy 0, policy_version 57650 (0.0007) -[2023-10-15 17:05:46,013][52833] Updated weights for policy 0, policy_version 57660 (0.0007) -[2023-10-15 17:05:48,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 118292480. Throughput: 0: 1782.5, 1: 1792.4. Samples: 29577968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:05:48,441][51532] Avg episode reward: [(0, '54.570'), (1, '56.210')] -[2023-10-15 17:05:49,128][52866] Updated weights for policy 1, policy_version 57860 (0.0008) -[2023-10-15 17:05:49,499][52866] Updated weights for policy 1, policy_version 57870 (0.0008) -[2023-10-15 17:05:49,824][52833] Updated weights for policy 0, policy_version 57670 (0.0009) -[2023-10-15 17:05:49,875][52866] Updated weights for policy 1, policy_version 57880 (0.0009) -[2023-10-15 17:05:50,197][52833] Updated weights for policy 0, policy_version 57680 (0.0009) -[2023-10-15 17:05:50,573][52833] Updated weights for policy 0, policy_version 57690 (0.0010) -[2023-10-15 17:05:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 118358016. Throughput: 0: 1770.0, 1: 1789.9. Samples: 29599822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:05:53,441][51532] Avg episode reward: [(0, '55.500'), (1, '55.090')] -[2023-10-15 17:05:53,691][52866] Updated weights for policy 1, policy_version 57890 (0.0009) -[2023-10-15 17:05:54,052][52866] Updated weights for policy 1, policy_version 57900 (0.0008) -[2023-10-15 17:05:54,275][52833] Updated weights for policy 0, policy_version 57700 (0.0009) -[2023-10-15 17:05:54,415][52866] Updated weights for policy 1, policy_version 57910 (0.0007) -[2023-10-15 17:05:54,645][52833] Updated weights for policy 0, policy_version 57710 (0.0008) -[2023-10-15 17:05:54,779][52866] Updated weights for policy 1, policy_version 57920 (0.0009) -[2023-10-15 17:05:55,012][52833] Updated weights for policy 0, policy_version 57720 (0.0007) -[2023-10-15 17:05:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 118423552. Throughput: 0: 1781.6, 1: 1806.2. Samples: 29622426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:05:58,441][51532] Avg episode reward: [(0, '55.410'), (1, '51.240')] -[2023-10-15 17:05:58,667][52833] Updated weights for policy 0, policy_version 57730 (0.0007) -[2023-10-15 17:05:58,706][52866] Updated weights for policy 1, policy_version 57930 (0.0008) -[2023-10-15 17:05:59,034][52833] Updated weights for policy 0, policy_version 57740 (0.0008) -[2023-10-15 17:05:59,069][52866] Updated weights for policy 1, policy_version 57940 (0.0007) -[2023-10-15 17:05:59,397][52833] Updated weights for policy 0, policy_version 57750 (0.0009) -[2023-10-15 17:05:59,431][52866] Updated weights for policy 1, policy_version 57950 (0.0007) -[2023-10-15 17:05:59,770][52833] Updated weights for policy 0, policy_version 57760 (0.0008) -[2023-10-15 17:06:03,070][52866] Updated weights for policy 1, policy_version 57960 (0.0008) -[2023-10-15 17:06:03,438][52866] Updated weights for policy 1, policy_version 57970 (0.0007) -[2023-10-15 17:06:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 118489088. Throughput: 0: 1771.9, 1: 1794.0. Samples: 29631900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:06:03,442][51532] Avg episode reward: [(0, '59.340'), (1, '52.300')] -[2023-10-15 17:06:03,542][52833] Updated weights for policy 0, policy_version 57770 (0.0007) -[2023-10-15 17:06:03,799][52866] Updated weights for policy 1, policy_version 57980 (0.0007) -[2023-10-15 17:06:03,913][52833] Updated weights for policy 0, policy_version 57780 (0.0007) -[2023-10-15 17:06:04,276][52833] Updated weights for policy 0, policy_version 57790 (0.0008) -[2023-10-15 17:06:07,498][52866] Updated weights for policy 1, policy_version 57990 (0.0008) -[2023-10-15 17:06:07,866][52866] Updated weights for policy 1, policy_version 58000 (0.0007) -[2023-10-15 17:06:08,037][52833] Updated weights for policy 0, policy_version 57800 (0.0008) -[2023-10-15 17:06:08,224][52866] Updated weights for policy 1, policy_version 58010 (0.0007) -[2023-10-15 17:06:08,410][52833] Updated weights for policy 0, policy_version 57810 (0.0009) -[2023-10-15 17:06:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 118554624. Throughput: 0: 1776.0, 1: 1803.3. Samples: 29654476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:06:08,442][51532] Avg episode reward: [(0, '59.920'), (1, '49.520')] -[2023-10-15 17:06:08,783][52833] Updated weights for policy 0, policy_version 57820 (0.0008) -[2023-10-15 17:06:12,016][52866] Updated weights for policy 1, policy_version 58020 (0.0009) -[2023-10-15 17:06:12,382][52866] Updated weights for policy 1, policy_version 58030 (0.0009) -[2023-10-15 17:06:12,609][52833] Updated weights for policy 0, policy_version 57830 (0.0007) -[2023-10-15 17:06:12,744][52866] Updated weights for policy 1, policy_version 58040 (0.0007) -[2023-10-15 17:06:12,975][52833] Updated weights for policy 0, policy_version 57840 (0.0008) -[2023-10-15 17:06:13,353][52833] Updated weights for policy 0, policy_version 57850 (0.0009) -[2023-10-15 17:06:13,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 118652928. Throughput: 0: 1792.0, 1: 1787.5. Samples: 29674760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:06:13,442][51532] Avg episode reward: [(0, '59.910'), (1, '49.980')] -[2023-10-15 17:06:16,563][52866] Updated weights for policy 1, policy_version 58050 (0.0008) -[2023-10-15 17:06:16,935][52866] Updated weights for policy 1, policy_version 58060 (0.0010) -[2023-10-15 17:06:17,198][52833] Updated weights for policy 0, policy_version 57860 (0.0009) -[2023-10-15 17:06:17,304][52866] Updated weights for policy 1, policy_version 58070 (0.0007) -[2023-10-15 17:06:17,574][52833] Updated weights for policy 0, policy_version 57870 (0.0008) -[2023-10-15 17:06:17,664][52866] Updated weights for policy 1, policy_version 58080 (0.0008) -[2023-10-15 17:06:17,936][52833] Updated weights for policy 0, policy_version 57880 (0.0008) -[2023-10-15 17:06:18,441][51532] Fps is (10 sec: 19660.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 118751232. Throughput: 0: 1775.2, 1: 1797.6. Samples: 29686394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:06:18,442][51532] Avg episode reward: [(0, '60.350'), (1, '50.160')] -[2023-10-15 17:06:21,270][52866] Updated weights for policy 1, policy_version 58090 (0.0009) -[2023-10-15 17:06:21,631][52833] Updated weights for policy 0, policy_version 57890 (0.0008) -[2023-10-15 17:06:21,638][52866] Updated weights for policy 1, policy_version 58100 (0.0007) -[2023-10-15 17:06:21,993][52833] Updated weights for policy 0, policy_version 57900 (0.0007) -[2023-10-15 17:06:22,008][52866] Updated weights for policy 1, policy_version 58110 (0.0007) -[2023-10-15 17:06:22,369][52833] Updated weights for policy 0, policy_version 57910 (0.0009) -[2023-10-15 17:06:22,733][52833] Updated weights for policy 0, policy_version 57920 (0.0007) -[2023-10-15 17:06:23,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118816768. Throughput: 0: 1801.4, 1: 1792.8. Samples: 29707196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:06:23,441][51532] Avg episode reward: [(0, '60.340'), (1, '45.960')] -[2023-10-15 17:06:25,876][52866] Updated weights for policy 1, policy_version 58120 (0.0009) -[2023-10-15 17:06:26,243][52866] Updated weights for policy 1, policy_version 58130 (0.0007) -[2023-10-15 17:06:26,589][52833] Updated weights for policy 0, policy_version 57930 (0.0009) -[2023-10-15 17:06:26,607][52866] Updated weights for policy 1, policy_version 58140 (0.0007) -[2023-10-15 17:06:26,961][52833] Updated weights for policy 0, policy_version 57940 (0.0008) -[2023-10-15 17:06:27,339][52833] Updated weights for policy 0, policy_version 57950 (0.0007) -[2023-10-15 17:06:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 118882304. Throughput: 0: 1778.2, 1: 1786.0. Samples: 29728132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:06:28,442][51532] Avg episode reward: [(0, '62.010'), (1, '47.680')] -[2023-10-15 17:06:30,242][52866] Updated weights for policy 1, policy_version 58150 (0.0007) -[2023-10-15 17:06:30,611][52866] Updated weights for policy 1, policy_version 58160 (0.0009) -[2023-10-15 17:06:30,981][52866] Updated weights for policy 1, policy_version 58170 (0.0009) -[2023-10-15 17:06:31,112][52833] Updated weights for policy 0, policy_version 57960 (0.0008) -[2023-10-15 17:06:31,480][52833] Updated weights for policy 0, policy_version 57970 (0.0008) -[2023-10-15 17:06:31,844][52833] Updated weights for policy 0, policy_version 57980 (0.0007) -[2023-10-15 17:06:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 118947840. Throughput: 0: 1796.6, 1: 1797.4. Samples: 29739698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:06:33,441][51532] Avg episode reward: [(0, '61.730'), (1, '45.800')] -[2023-10-15 17:06:34,848][52866] Updated weights for policy 1, policy_version 58180 (0.0010) -[2023-10-15 17:06:35,205][52866] Updated weights for policy 1, policy_version 58190 (0.0009) -[2023-10-15 17:06:35,581][52866] Updated weights for policy 1, policy_version 58200 (0.0009) -[2023-10-15 17:06:35,687][52833] Updated weights for policy 0, policy_version 57990 (0.0007) -[2023-10-15 17:06:36,068][52833] Updated weights for policy 0, policy_version 58000 (0.0007) -[2023-10-15 17:06:36,430][52833] Updated weights for policy 0, policy_version 58010 (0.0007) -[2023-10-15 17:06:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 119013376. Throughput: 0: 1777.1, 1: 1792.6. Samples: 29760460. Policy #0 lag: (min: 6.0, avg: 7.4, max: 32.0) -[2023-10-15 17:06:38,442][51532] Avg episode reward: [(0, '59.090'), (1, '47.670')] -[2023-10-15 17:06:39,395][52866] Updated weights for policy 1, policy_version 58210 (0.0010) -[2023-10-15 17:06:39,764][52866] Updated weights for policy 1, policy_version 58220 (0.0011) -[2023-10-15 17:06:40,034][52833] Updated weights for policy 0, policy_version 58020 (0.0008) -[2023-10-15 17:06:40,132][52866] Updated weights for policy 1, policy_version 58230 (0.0008) -[2023-10-15 17:06:40,388][52833] Updated weights for policy 0, policy_version 58030 (0.0008) -[2023-10-15 17:06:40,493][52866] Updated weights for policy 1, policy_version 58240 (0.0009) -[2023-10-15 17:06:40,758][52833] Updated weights for policy 0, policy_version 58040 (0.0008) -[2023-10-15 17:06:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 119078912. Throughput: 0: 1773.5, 1: 1795.3. Samples: 29783024. Policy #0 lag: (min: 6.0, avg: 7.4, max: 32.0) -[2023-10-15 17:06:43,442][51532] Avg episode reward: [(0, '57.050'), (1, '47.040')] -[2023-10-15 17:06:44,263][52866] Updated weights for policy 1, policy_version 58250 (0.0008) -[2023-10-15 17:06:44,614][52833] Updated weights for policy 0, policy_version 58050 (0.0008) -[2023-10-15 17:06:44,634][52866] Updated weights for policy 1, policy_version 58260 (0.0007) -[2023-10-15 17:06:44,990][52833] Updated weights for policy 0, policy_version 58060 (0.0008) -[2023-10-15 17:06:45,003][52866] Updated weights for policy 1, policy_version 58270 (0.0008) -[2023-10-15 17:06:45,351][52833] Updated weights for policy 0, policy_version 58070 (0.0009) -[2023-10-15 17:06:45,723][52833] Updated weights for policy 0, policy_version 58080 (0.0008) -[2023-10-15 17:06:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 119144448. Throughput: 0: 1779.5, 1: 1794.9. Samples: 29792750. Policy #0 lag: (min: 6.0, avg: 7.4, max: 32.0) -[2023-10-15 17:06:48,441][51532] Avg episode reward: [(0, '57.020'), (1, '47.000')] -[2023-10-15 17:06:48,786][52866] Updated weights for policy 1, policy_version 58280 (0.0009) -[2023-10-15 17:06:49,156][52866] Updated weights for policy 1, policy_version 58290 (0.0008) -[2023-10-15 17:06:49,402][52833] Updated weights for policy 0, policy_version 58090 (0.0008) -[2023-10-15 17:06:49,518][52866] Updated weights for policy 1, policy_version 58300 (0.0008) -[2023-10-15 17:06:49,786][52833] Updated weights for policy 0, policy_version 58100 (0.0009) -[2023-10-15 17:06:50,152][52833] Updated weights for policy 0, policy_version 58110 (0.0008) -[2023-10-15 17:06:53,227][52866] Updated weights for policy 1, policy_version 58310 (0.0009) -[2023-10-15 17:06:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 119209984. Throughput: 0: 1781.7, 1: 1790.2. Samples: 29815212. Policy #0 lag: (min: 6.0, avg: 7.4, max: 32.0) -[2023-10-15 17:06:53,441][51532] Avg episode reward: [(0, '54.580'), (1, '47.810')] -[2023-10-15 17:06:53,601][52866] Updated weights for policy 1, policy_version 58320 (0.0009) -[2023-10-15 17:06:53,865][52833] Updated weights for policy 0, policy_version 58120 (0.0009) -[2023-10-15 17:06:53,970][52866] Updated weights for policy 1, policy_version 58330 (0.0009) -[2023-10-15 17:06:54,233][52833] Updated weights for policy 0, policy_version 58130 (0.0008) -[2023-10-15 17:06:54,604][52833] Updated weights for policy 0, policy_version 58140 (0.0007) -[2023-10-15 17:06:57,629][52866] Updated weights for policy 1, policy_version 58340 (0.0007) -[2023-10-15 17:06:57,997][52866] Updated weights for policy 1, policy_version 58350 (0.0010) -[2023-10-15 17:06:58,368][52866] Updated weights for policy 1, policy_version 58360 (0.0009) -[2023-10-15 17:06:58,437][52833] Updated weights for policy 0, policy_version 58150 (0.0007) -[2023-10-15 17:06:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 119275520. Throughput: 0: 1797.2, 1: 1814.4. Samples: 29837280. Policy #0 lag: (min: 6.0, avg: 7.4, max: 32.0) -[2023-10-15 17:06:58,441][51532] Avg episode reward: [(0, '49.080'), (1, '50.860')] -[2023-10-15 17:06:58,816][52833] Updated weights for policy 0, policy_version 58160 (0.0009) -[2023-10-15 17:06:59,179][52833] Updated weights for policy 0, policy_version 58170 (0.0008) -[2023-10-15 17:07:01,985][52866] Updated weights for policy 1, policy_version 58370 (0.0008) -[2023-10-15 17:07:02,356][52866] Updated weights for policy 1, policy_version 58380 (0.0008) -[2023-10-15 17:07:02,721][52866] Updated weights for policy 1, policy_version 58390 (0.0007) -[2023-10-15 17:07:02,888][52833] Updated weights for policy 0, policy_version 58180 (0.0007) -[2023-10-15 17:07:03,092][52866] Updated weights for policy 1, policy_version 58400 (0.0008) -[2023-10-15 17:07:03,263][52833] Updated weights for policy 0, policy_version 58190 (0.0009) -[2023-10-15 17:07:03,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 119373824. Throughput: 0: 1784.5, 1: 1796.2. Samples: 29847526. Policy #0 lag: (min: 6.0, avg: 7.4, max: 32.0) -[2023-10-15 17:07:03,442][51532] Avg episode reward: [(0, '47.950'), (1, '51.440')] -[2023-10-15 17:07:03,627][52833] Updated weights for policy 0, policy_version 58200 (0.0007) -[2023-10-15 17:07:06,719][52866] Updated weights for policy 1, policy_version 58410 (0.0008) -[2023-10-15 17:07:07,087][52866] Updated weights for policy 1, policy_version 58420 (0.0008) -[2023-10-15 17:07:07,336][52833] Updated weights for policy 0, policy_version 58210 (0.0008) -[2023-10-15 17:07:07,456][52866] Updated weights for policy 1, policy_version 58430 (0.0007) -[2023-10-15 17:07:07,697][52833] Updated weights for policy 0, policy_version 58220 (0.0008) -[2023-10-15 17:07:08,067][52833] Updated weights for policy 0, policy_version 58230 (0.0009) -[2023-10-15 17:07:08,438][52833] Updated weights for policy 0, policy_version 58240 (0.0009) -[2023-10-15 17:07:08,441][51532] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14440.2). Total num frames: 119472128. Throughput: 0: 1795.6, 1: 1806.8. Samples: 29869304. Policy #0 lag: (min: 6.0, avg: 7.4, max: 32.0) -[2023-10-15 17:07:08,441][51532] Avg episode reward: [(0, '51.260'), (1, '52.360')] -[2023-10-15 17:07:11,264][52866] Updated weights for policy 1, policy_version 58440 (0.0008) -[2023-10-15 17:07:11,621][52866] Updated weights for policy 1, policy_version 58450 (0.0012) -[2023-10-15 17:07:11,982][52866] Updated weights for policy 1, policy_version 58460 (0.0008) -[2023-10-15 17:07:12,186][52833] Updated weights for policy 0, policy_version 58250 (0.0007) -[2023-10-15 17:07:12,564][52833] Updated weights for policy 0, policy_version 58260 (0.0010) -[2023-10-15 17:07:12,936][52833] Updated weights for policy 0, policy_version 58270 (0.0008) -[2023-10-15 17:07:13,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 119537664. Throughput: 0: 1793.7, 1: 1798.2. Samples: 29889764. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:13,441][51532] Avg episode reward: [(0, '52.110'), (1, '54.080')] -[2023-10-15 17:07:15,661][52866] Updated weights for policy 1, policy_version 58470 (0.0009) -[2023-10-15 17:07:16,030][52866] Updated weights for policy 1, policy_version 58480 (0.0009) -[2023-10-15 17:07:16,392][52866] Updated weights for policy 1, policy_version 58490 (0.0011) -[2023-10-15 17:07:16,885][52833] Updated weights for policy 0, policy_version 58280 (0.0008) -[2023-10-15 17:07:17,252][52833] Updated weights for policy 0, policy_version 58290 (0.0008) -[2023-10-15 17:07:17,613][52833] Updated weights for policy 0, policy_version 58300 (0.0008) -[2023-10-15 17:07:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 119603200. Throughput: 0: 1787.3, 1: 1809.1. Samples: 29901536. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:18,441][51532] Avg episode reward: [(0, '51.970'), (1, '51.820')] -[2023-10-15 17:07:20,107][52866] Updated weights for policy 1, policy_version 58500 (0.0008) -[2023-10-15 17:07:20,481][52866] Updated weights for policy 1, policy_version 58510 (0.0008) -[2023-10-15 17:07:20,839][52866] Updated weights for policy 1, policy_version 58520 (0.0010) -[2023-10-15 17:07:21,494][52833] Updated weights for policy 0, policy_version 58310 (0.0008) -[2023-10-15 17:07:21,873][52833] Updated weights for policy 0, policy_version 58320 (0.0009) -[2023-10-15 17:07:22,242][52833] Updated weights for policy 0, policy_version 58330 (0.0008) -[2023-10-15 17:07:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 119668736. Throughput: 0: 1800.7, 1: 1796.8. Samples: 29922346. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:23,442][51532] Avg episode reward: [(0, '53.680'), (1, '52.590')] -[2023-10-15 17:07:24,475][52866] Updated weights for policy 1, policy_version 58530 (0.0008) -[2023-10-15 17:07:24,851][52866] Updated weights for policy 1, policy_version 58540 (0.0009) -[2023-10-15 17:07:25,214][52866] Updated weights for policy 1, policy_version 58550 (0.0009) -[2023-10-15 17:07:25,583][52866] Updated weights for policy 1, policy_version 58560 (0.0009) -[2023-10-15 17:07:25,918][52833] Updated weights for policy 0, policy_version 58340 (0.0009) -[2023-10-15 17:07:26,295][52833] Updated weights for policy 0, policy_version 58350 (0.0008) -[2023-10-15 17:07:26,662][52833] Updated weights for policy 0, policy_version 58360 (0.0008) -[2023-10-15 17:07:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 119734272. Throughput: 0: 1786.3, 1: 1799.7. Samples: 29944392. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:28,441][51532] Avg episode reward: [(0, '55.430'), (1, '52.200')] -[2023-10-15 17:07:29,443][52866] Updated weights for policy 1, policy_version 58570 (0.0007) -[2023-10-15 17:07:29,814][52866] Updated weights for policy 1, policy_version 58580 (0.0007) -[2023-10-15 17:07:30,183][52866] Updated weights for policy 1, policy_version 58590 (0.0008) -[2023-10-15 17:07:30,446][52833] Updated weights for policy 0, policy_version 58370 (0.0007) -[2023-10-15 17:07:30,814][52833] Updated weights for policy 0, policy_version 58380 (0.0008) -[2023-10-15 17:07:31,184][52833] Updated weights for policy 0, policy_version 58390 (0.0011) -[2023-10-15 17:07:31,557][52833] Updated weights for policy 0, policy_version 58400 (0.0009) -[2023-10-15 17:07:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 119799808. Throughput: 0: 1803.5, 1: 1801.2. Samples: 29954962. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:33,442][51532] Avg episode reward: [(0, '52.300'), (1, '53.470')] -[2023-10-15 17:07:33,880][52866] Updated weights for policy 1, policy_version 58600 (0.0010) -[2023-10-15 17:07:34,245][52866] Updated weights for policy 1, policy_version 58610 (0.0009) -[2023-10-15 17:07:34,606][52866] Updated weights for policy 1, policy_version 58620 (0.0011) -[2023-10-15 17:07:35,075][52833] Updated weights for policy 0, policy_version 58410 (0.0008) -[2023-10-15 17:07:35,447][52833] Updated weights for policy 0, policy_version 58420 (0.0008) -[2023-10-15 17:07:35,812][52833] Updated weights for policy 0, policy_version 58430 (0.0008) -[2023-10-15 17:07:38,366][52866] Updated weights for policy 1, policy_version 58630 (0.0008) -[2023-10-15 17:07:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 119865344. Throughput: 0: 1785.6, 1: 1812.0. Samples: 29977104. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:38,442][51532] Avg episode reward: [(0, '52.750'), (1, '53.390')] -[2023-10-15 17:07:38,736][52866] Updated weights for policy 1, policy_version 58640 (0.0008) -[2023-10-15 17:07:39,099][52866] Updated weights for policy 1, policy_version 58650 (0.0008) -[2023-10-15 17:07:39,590][52833] Updated weights for policy 0, policy_version 58440 (0.0008) -[2023-10-15 17:07:39,965][52833] Updated weights for policy 0, policy_version 58450 (0.0008) -[2023-10-15 17:07:40,339][52833] Updated weights for policy 0, policy_version 58460 (0.0008) -[2023-10-15 17:07:42,786][52866] Updated weights for policy 1, policy_version 58660 (0.0008) -[2023-10-15 17:07:43,146][52866] Updated weights for policy 1, policy_version 58670 (0.0010) -[2023-10-15 17:07:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 119930880. Throughput: 0: 1787.3, 1: 1812.4. Samples: 29999270. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:43,441][51532] Avg episode reward: [(0, '54.530'), (1, '53.450')] -[2023-10-15 17:07:43,448][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000058464_59867136.pth... -[2023-10-15 17:07:43,487][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000056800_58163200.pth -[2023-10-15 17:07:43,491][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000058464_59867136.pth -[2023-10-15 17:07:43,519][52866] Updated weights for policy 1, policy_version 58680 (0.0008) -[2023-10-15 17:07:43,805][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000058688_60096512.pth... -[2023-10-15 17:07:43,846][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000056992_58359808.pth -[2023-10-15 17:07:43,851][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000058688_60096512.pth -[2023-10-15 17:07:44,003][52833] Updated weights for policy 0, policy_version 58470 (0.0011) -[2023-10-15 17:07:44,371][52833] Updated weights for policy 0, policy_version 58480 (0.0008) -[2023-10-15 17:07:44,741][52833] Updated weights for policy 0, policy_version 58490 (0.0007) -[2023-10-15 17:07:47,292][52866] Updated weights for policy 1, policy_version 58690 (0.0007) -[2023-10-15 17:07:47,654][52866] Updated weights for policy 1, policy_version 58700 (0.0010) -[2023-10-15 17:07:48,023][52866] Updated weights for policy 1, policy_version 58710 (0.0011) -[2023-10-15 17:07:48,383][52866] Updated weights for policy 1, policy_version 58720 (0.0008) -[2023-10-15 17:07:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 120029184. Throughput: 0: 1790.1, 1: 1809.7. Samples: 30009516. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:48,441][51532] Avg episode reward: [(0, '56.440'), (1, '53.530')] -[2023-10-15 17:07:48,479][52833] Updated weights for policy 0, policy_version 58500 (0.0008) -[2023-10-15 17:07:48,859][52833] Updated weights for policy 0, policy_version 58510 (0.0009) -[2023-10-15 17:07:49,232][52833] Updated weights for policy 0, policy_version 58520 (0.0008) -[2023-10-15 17:07:51,849][52866] Updated weights for policy 1, policy_version 58730 (0.0008) -[2023-10-15 17:07:52,220][52866] Updated weights for policy 1, policy_version 58740 (0.0007) -[2023-10-15 17:07:52,587][52866] Updated weights for policy 1, policy_version 58750 (0.0007) -[2023-10-15 17:07:53,011][52833] Updated weights for policy 0, policy_version 58530 (0.0008) -[2023-10-15 17:07:53,379][52833] Updated weights for policy 0, policy_version 58540 (0.0008) -[2023-10-15 17:07:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 120094720. Throughput: 0: 1787.7, 1: 1818.9. Samples: 30031604. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) -[2023-10-15 17:07:53,441][51532] Avg episode reward: [(0, '57.200'), (1, '54.140')] -[2023-10-15 17:07:53,753][52833] Updated weights for policy 0, policy_version 58550 (0.0009) -[2023-10-15 17:07:54,124][52833] Updated weights for policy 0, policy_version 58560 (0.0009) -[2023-10-15 17:07:56,382][52866] Updated weights for policy 1, policy_version 58760 (0.0010) -[2023-10-15 17:07:56,748][52866] Updated weights for policy 1, policy_version 58770 (0.0009) -[2023-10-15 17:07:57,119][52866] Updated weights for policy 1, policy_version 58780 (0.0010) -[2023-10-15 17:07:57,880][52833] Updated weights for policy 0, policy_version 58570 (0.0009) -[2023-10-15 17:07:58,249][52833] Updated weights for policy 0, policy_version 58580 (0.0007) -[2023-10-15 17:07:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 120160256. Throughput: 0: 1810.1, 1: 1812.9. Samples: 30052800. Policy #0 lag: (min: 21.0, avg: 21.0, max: 23.0) -[2023-10-15 17:07:58,441][51532] Avg episode reward: [(0, '54.650'), (1, '51.910')] -[2023-10-15 17:07:58,625][52833] Updated weights for policy 0, policy_version 58590 (0.0007) -[2023-10-15 17:08:00,685][52866] Updated weights for policy 1, policy_version 58790 (0.0009) -[2023-10-15 17:08:01,054][52866] Updated weights for policy 1, policy_version 58800 (0.0009) -[2023-10-15 17:08:01,419][52866] Updated weights for policy 1, policy_version 58810 (0.0008) -[2023-10-15 17:08:02,335][52833] Updated weights for policy 0, policy_version 58600 (0.0009) -[2023-10-15 17:08:02,699][52833] Updated weights for policy 0, policy_version 58610 (0.0007) -[2023-10-15 17:08:03,069][52833] Updated weights for policy 0, policy_version 58620 (0.0007) -[2023-10-15 17:08:03,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 120258560. Throughput: 0: 1796.2, 1: 1809.6. Samples: 30063796. Policy #0 lag: (min: 21.0, avg: 21.0, max: 23.0) -[2023-10-15 17:08:03,442][51532] Avg episode reward: [(0, '55.860'), (1, '50.430')] -[2023-10-15 17:08:05,082][52866] Updated weights for policy 1, policy_version 58820 (0.0007) -[2023-10-15 17:08:05,450][52866] Updated weights for policy 1, policy_version 58830 (0.0009) -[2023-10-15 17:08:05,826][52866] Updated weights for policy 1, policy_version 58840 (0.0008) -[2023-10-15 17:08:06,839][52833] Updated weights for policy 0, policy_version 58630 (0.0007) -[2023-10-15 17:08:07,202][52833] Updated weights for policy 0, policy_version 58640 (0.0010) -[2023-10-15 17:08:07,571][52833] Updated weights for policy 0, policy_version 58650 (0.0010) -[2023-10-15 17:08:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 120324096. Throughput: 0: 1811.4, 1: 1813.7. Samples: 30085478. Policy #0 lag: (min: 21.0, avg: 21.0, max: 23.0) -[2023-10-15 17:08:08,442][51532] Avg episode reward: [(0, '57.280'), (1, '52.090')] -[2023-10-15 17:08:09,623][52866] Updated weights for policy 1, policy_version 58850 (0.0008) -[2023-10-15 17:08:10,001][52866] Updated weights for policy 1, policy_version 58860 (0.0008) -[2023-10-15 17:08:10,364][52866] Updated weights for policy 1, policy_version 58870 (0.0008) -[2023-10-15 17:08:10,724][52866] Updated weights for policy 1, policy_version 58880 (0.0007) -[2023-10-15 17:08:11,394][52833] Updated weights for policy 0, policy_version 58660 (0.0007) -[2023-10-15 17:08:11,762][52833] Updated weights for policy 0, policy_version 58670 (0.0008) -[2023-10-15 17:08:12,131][52833] Updated weights for policy 0, policy_version 58680 (0.0007) -[2023-10-15 17:08:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 120389632. Throughput: 0: 1800.4, 1: 1806.2. Samples: 30106688. Policy #0 lag: (min: 21.0, avg: 21.0, max: 23.0) -[2023-10-15 17:08:13,441][51532] Avg episode reward: [(0, '56.050'), (1, '50.390')] -[2023-10-15 17:08:14,652][52866] Updated weights for policy 1, policy_version 58890 (0.0010) -[2023-10-15 17:08:15,024][52866] Updated weights for policy 1, policy_version 58900 (0.0010) -[2023-10-15 17:08:15,386][52866] Updated weights for policy 1, policy_version 58910 (0.0010) -[2023-10-15 17:08:15,962][52833] Updated weights for policy 0, policy_version 58690 (0.0008) -[2023-10-15 17:08:16,333][52833] Updated weights for policy 0, policy_version 58700 (0.0008) -[2023-10-15 17:08:16,702][52833] Updated weights for policy 0, policy_version 58710 (0.0009) -[2023-10-15 17:08:17,070][52833] Updated weights for policy 0, policy_version 58720 (0.0008) -[2023-10-15 17:08:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 120455168. Throughput: 0: 1814.8, 1: 1804.1. Samples: 30117812. Policy #0 lag: (min: 21.0, avg: 21.0, max: 23.0) -[2023-10-15 17:08:18,442][51532] Avg episode reward: [(0, '53.690'), (1, '48.700')] -[2023-10-15 17:08:19,225][52866] Updated weights for policy 1, policy_version 58920 (0.0008) -[2023-10-15 17:08:19,592][52866] Updated weights for policy 1, policy_version 58930 (0.0008) -[2023-10-15 17:08:19,952][52866] Updated weights for policy 1, policy_version 58940 (0.0009) -[2023-10-15 17:08:20,795][52833] Updated weights for policy 0, policy_version 58730 (0.0009) -[2023-10-15 17:08:21,161][52833] Updated weights for policy 0, policy_version 58740 (0.0008) -[2023-10-15 17:08:21,535][52833] Updated weights for policy 0, policy_version 58750 (0.0008) -[2023-10-15 17:08:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 120520704. Throughput: 0: 1799.0, 1: 1798.6. Samples: 30138994. Policy #0 lag: (min: 21.0, avg: 21.0, max: 23.0) -[2023-10-15 17:08:23,441][51532] Avg episode reward: [(0, '49.720'), (1, '46.650')] -[2023-10-15 17:08:23,732][52866] Updated weights for policy 1, policy_version 58950 (0.0010) -[2023-10-15 17:08:24,085][52866] Updated weights for policy 1, policy_version 58960 (0.0010) -[2023-10-15 17:08:24,452][52866] Updated weights for policy 1, policy_version 58970 (0.0007) -[2023-10-15 17:08:25,268][52833] Updated weights for policy 0, policy_version 58760 (0.0008) -[2023-10-15 17:08:25,643][52833] Updated weights for policy 0, policy_version 58770 (0.0009) -[2023-10-15 17:08:26,014][52833] Updated weights for policy 0, policy_version 58780 (0.0010) -[2023-10-15 17:08:28,084][52866] Updated weights for policy 1, policy_version 58980 (0.0008) -[2023-10-15 17:08:28,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 120586240. Throughput: 0: 1796.6, 1: 1810.1. Samples: 30161570. Policy #0 lag: (min: 21.0, avg: 21.0, max: 23.0) -[2023-10-15 17:08:28,441][51532] Avg episode reward: [(0, '53.000'), (1, '48.040')] -[2023-10-15 17:08:28,448][52866] Updated weights for policy 1, policy_version 58990 (0.0007) -[2023-10-15 17:08:28,811][52866] Updated weights for policy 1, policy_version 59000 (0.0009) -[2023-10-15 17:08:29,702][52833] Updated weights for policy 0, policy_version 58790 (0.0010) -[2023-10-15 17:08:30,061][52833] Updated weights for policy 0, policy_version 58800 (0.0007) -[2023-10-15 17:08:30,430][52833] Updated weights for policy 0, policy_version 58810 (0.0007) -[2023-10-15 17:08:32,668][52866] Updated weights for policy 1, policy_version 59010 (0.0009) -[2023-10-15 17:08:33,034][52866] Updated weights for policy 1, policy_version 59020 (0.0007) -[2023-10-15 17:08:33,403][52866] Updated weights for policy 1, policy_version 59030 (0.0008) -[2023-10-15 17:08:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 120651776. Throughput: 0: 1800.7, 1: 1803.8. Samples: 30171718. Policy #0 lag: (min: 21.0, avg: 21.0, max: 23.0) -[2023-10-15 17:08:33,441][51532] Avg episode reward: [(0, '54.590'), (1, '48.500')] -[2023-10-15 17:08:33,775][52866] Updated weights for policy 1, policy_version 59040 (0.0007) -[2023-10-15 17:08:34,110][52833] Updated weights for policy 0, policy_version 58820 (0.0010) -[2023-10-15 17:08:34,473][52833] Updated weights for policy 0, policy_version 58830 (0.0009) -[2023-10-15 17:08:34,840][52833] Updated weights for policy 0, policy_version 58840 (0.0009) -[2023-10-15 17:08:37,523][52866] Updated weights for policy 1, policy_version 59050 (0.0008) -[2023-10-15 17:08:37,884][52866] Updated weights for policy 1, policy_version 59060 (0.0008) -[2023-10-15 17:08:38,246][52866] Updated weights for policy 1, policy_version 59070 (0.0010) -[2023-10-15 17:08:38,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 120750080. Throughput: 0: 1804.7, 1: 1809.3. Samples: 30194234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:08:38,442][51532] Avg episode reward: [(0, '53.140'), (1, '48.440')] -[2023-10-15 17:08:38,633][52833] Updated weights for policy 0, policy_version 58850 (0.0009) -[2023-10-15 17:08:38,993][52833] Updated weights for policy 0, policy_version 58860 (0.0008) -[2023-10-15 17:08:39,367][52833] Updated weights for policy 0, policy_version 58870 (0.0007) -[2023-10-15 17:08:39,736][52833] Updated weights for policy 0, policy_version 58880 (0.0007) -[2023-10-15 17:08:42,039][52866] Updated weights for policy 1, policy_version 59080 (0.0008) -[2023-10-15 17:08:42,411][52866] Updated weights for policy 1, policy_version 59090 (0.0007) -[2023-10-15 17:08:42,778][52866] Updated weights for policy 1, policy_version 59100 (0.0008) -[2023-10-15 17:08:43,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 120815616. Throughput: 0: 1810.8, 1: 1799.6. Samples: 30215270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:08:43,441][51532] Avg episode reward: [(0, '53.840'), (1, '47.910')] -[2023-10-15 17:08:43,621][52833] Updated weights for policy 0, policy_version 58890 (0.0009) -[2023-10-15 17:08:43,991][52833] Updated weights for policy 0, policy_version 58900 (0.0007) -[2023-10-15 17:08:44,359][52833] Updated weights for policy 0, policy_version 58910 (0.0008) -[2023-10-15 17:08:46,639][52866] Updated weights for policy 1, policy_version 59110 (0.0010) -[2023-10-15 17:08:47,013][52866] Updated weights for policy 1, policy_version 59120 (0.0011) -[2023-10-15 17:08:47,388][52866] Updated weights for policy 1, policy_version 59130 (0.0009) -[2023-10-15 17:08:48,009][52833] Updated weights for policy 0, policy_version 58920 (0.0009) -[2023-10-15 17:08:48,376][52833] Updated weights for policy 0, policy_version 58930 (0.0008) -[2023-10-15 17:08:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 120881152. Throughput: 0: 1799.3, 1: 1814.5. Samples: 30226418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:08:48,441][51532] Avg episode reward: [(0, '51.880'), (1, '47.110')] -[2023-10-15 17:08:48,741][52833] Updated weights for policy 0, policy_version 58940 (0.0010) -[2023-10-15 17:08:51,092][52866] Updated weights for policy 1, policy_version 59140 (0.0011) -[2023-10-15 17:08:51,451][52866] Updated weights for policy 1, policy_version 59150 (0.0009) -[2023-10-15 17:08:51,810][52866] Updated weights for policy 1, policy_version 59160 (0.0010) -[2023-10-15 17:08:52,475][52833] Updated weights for policy 0, policy_version 58950 (0.0009) -[2023-10-15 17:08:52,842][52833] Updated weights for policy 0, policy_version 58960 (0.0008) -[2023-10-15 17:08:53,217][52833] Updated weights for policy 0, policy_version 58970 (0.0009) -[2023-10-15 17:08:53,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 120979456. Throughput: 0: 1804.1, 1: 1799.0. Samples: 30247616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:08:53,442][51532] Avg episode reward: [(0, '52.910'), (1, '50.260')] -[2023-10-15 17:08:55,393][52866] Updated weights for policy 1, policy_version 59170 (0.0010) -[2023-10-15 17:08:55,759][52866] Updated weights for policy 1, policy_version 59180 (0.0010) -[2023-10-15 17:08:56,116][52866] Updated weights for policy 1, policy_version 59190 (0.0008) -[2023-10-15 17:08:56,481][52866] Updated weights for policy 1, policy_version 59200 (0.0007) -[2023-10-15 17:08:56,926][52833] Updated weights for policy 0, policy_version 58980 (0.0008) -[2023-10-15 17:08:57,284][52833] Updated weights for policy 0, policy_version 58990 (0.0008) -[2023-10-15 17:08:57,662][52833] Updated weights for policy 0, policy_version 59000 (0.0007) -[2023-10-15 17:08:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 121044992. Throughput: 0: 1802.4, 1: 1801.9. Samples: 30268884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:08:58,442][51532] Avg episode reward: [(0, '50.420'), (1, '50.400')] -[2023-10-15 17:09:00,169][52866] Updated weights for policy 1, policy_version 59210 (0.0007) -[2023-10-15 17:09:00,543][52866] Updated weights for policy 1, policy_version 59220 (0.0007) -[2023-10-15 17:09:00,908][52866] Updated weights for policy 1, policy_version 59230 (0.0007) -[2023-10-15 17:09:01,420][52833] Updated weights for policy 0, policy_version 59010 (0.0009) -[2023-10-15 17:09:01,783][52833] Updated weights for policy 0, policy_version 59020 (0.0009) -[2023-10-15 17:09:02,161][52833] Updated weights for policy 0, policy_version 59030 (0.0010) -[2023-10-15 17:09:02,524][52833] Updated weights for policy 0, policy_version 59040 (0.0008) -[2023-10-15 17:09:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 121110528. Throughput: 0: 1801.1, 1: 1805.5. Samples: 30280106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:03,442][51532] Avg episode reward: [(0, '51.540'), (1, '50.730')] -[2023-10-15 17:09:04,661][52866] Updated weights for policy 1, policy_version 59240 (0.0008) -[2023-10-15 17:09:05,020][52866] Updated weights for policy 1, policy_version 59250 (0.0009) -[2023-10-15 17:09:05,389][52866] Updated weights for policy 1, policy_version 59260 (0.0007) -[2023-10-15 17:09:06,189][52833] Updated weights for policy 0, policy_version 59050 (0.0009) -[2023-10-15 17:09:06,560][52833] Updated weights for policy 0, policy_version 59060 (0.0008) -[2023-10-15 17:09:06,921][52833] Updated weights for policy 0, policy_version 59070 (0.0008) -[2023-10-15 17:09:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 121176064. Throughput: 0: 1807.2, 1: 1798.7. Samples: 30301262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:08,441][51532] Avg episode reward: [(0, '54.630'), (1, '52.060')] -[2023-10-15 17:09:08,984][52866] Updated weights for policy 1, policy_version 59270 (0.0007) -[2023-10-15 17:09:09,360][52866] Updated weights for policy 1, policy_version 59280 (0.0007) -[2023-10-15 17:09:09,720][52866] Updated weights for policy 1, policy_version 59290 (0.0008) -[2023-10-15 17:09:10,630][52833] Updated weights for policy 0, policy_version 59080 (0.0008) -[2023-10-15 17:09:11,004][52833] Updated weights for policy 0, policy_version 59090 (0.0007) -[2023-10-15 17:09:11,375][52833] Updated weights for policy 0, policy_version 59100 (0.0008) -[2023-10-15 17:09:13,326][52866] Updated weights for policy 1, policy_version 59300 (0.0008) -[2023-10-15 17:09:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 121241600. Throughput: 0: 1803.0, 1: 1805.9. Samples: 30323970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:13,441][51532] Avg episode reward: [(0, '57.680'), (1, '51.690')] -[2023-10-15 17:09:13,690][52866] Updated weights for policy 1, policy_version 59310 (0.0010) -[2023-10-15 17:09:14,062][52866] Updated weights for policy 1, policy_version 59320 (0.0008) -[2023-10-15 17:09:15,120][52833] Updated weights for policy 0, policy_version 59110 (0.0007) -[2023-10-15 17:09:15,490][52833] Updated weights for policy 0, policy_version 59120 (0.0008) -[2023-10-15 17:09:15,865][52833] Updated weights for policy 0, policy_version 59130 (0.0007) -[2023-10-15 17:09:17,920][52866] Updated weights for policy 1, policy_version 59330 (0.0009) -[2023-10-15 17:09:18,289][52866] Updated weights for policy 1, policy_version 59340 (0.0009) -[2023-10-15 17:09:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 121307136. Throughput: 0: 1807.6, 1: 1802.4. Samples: 30334170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:18,442][51532] Avg episode reward: [(0, '58.940'), (1, '54.140')] -[2023-10-15 17:09:18,661][52866] Updated weights for policy 1, policy_version 59350 (0.0009) -[2023-10-15 17:09:19,032][52866] Updated weights for policy 1, policy_version 59360 (0.0009) -[2023-10-15 17:09:19,531][52833] Updated weights for policy 0, policy_version 59140 (0.0008) -[2023-10-15 17:09:19,888][52833] Updated weights for policy 0, policy_version 59150 (0.0008) -[2023-10-15 17:09:20,258][52833] Updated weights for policy 0, policy_version 59160 (0.0009) -[2023-10-15 17:09:22,773][52866] Updated weights for policy 1, policy_version 59370 (0.0008) -[2023-10-15 17:09:23,147][52866] Updated weights for policy 1, policy_version 59380 (0.0010) -[2023-10-15 17:09:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 121372672. Throughput: 0: 1797.3, 1: 1801.4. Samples: 30356176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:23,441][51532] Avg episode reward: [(0, '55.670'), (1, '57.080')] -[2023-10-15 17:09:23,510][52866] Updated weights for policy 1, policy_version 59390 (0.0008) -[2023-10-15 17:09:24,026][52833] Updated weights for policy 0, policy_version 59170 (0.0010) -[2023-10-15 17:09:24,402][52833] Updated weights for policy 0, policy_version 59180 (0.0008) -[2023-10-15 17:09:24,776][52833] Updated weights for policy 0, policy_version 59190 (0.0009) -[2023-10-15 17:09:25,146][52833] Updated weights for policy 0, policy_version 59200 (0.0008) -[2023-10-15 17:09:27,225][52866] Updated weights for policy 1, policy_version 59400 (0.0007) -[2023-10-15 17:09:27,598][52866] Updated weights for policy 1, policy_version 59410 (0.0009) -[2023-10-15 17:09:27,971][52866] Updated weights for policy 1, policy_version 59420 (0.0008) -[2023-10-15 17:09:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 121470976. Throughput: 0: 1802.6, 1: 1804.7. Samples: 30377602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:28,442][51532] Avg episode reward: [(0, '55.760'), (1, '56.020')] -[2023-10-15 17:09:28,934][52833] Updated weights for policy 0, policy_version 59210 (0.0009) -[2023-10-15 17:09:29,298][52833] Updated weights for policy 0, policy_version 59220 (0.0008) -[2023-10-15 17:09:29,667][52833] Updated weights for policy 0, policy_version 59230 (0.0008) -[2023-10-15 17:09:31,741][52866] Updated weights for policy 1, policy_version 59430 (0.0010) -[2023-10-15 17:09:32,109][52866] Updated weights for policy 1, policy_version 59440 (0.0009) -[2023-10-15 17:09:32,475][52866] Updated weights for policy 1, policy_version 59450 (0.0011) -[2023-10-15 17:09:33,341][52833] Updated weights for policy 0, policy_version 59240 (0.0009) -[2023-10-15 17:09:33,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 121536512. Throughput: 0: 1801.6, 1: 1800.5. Samples: 30388514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:33,442][51532] Avg episode reward: [(0, '54.880'), (1, '54.920')] -[2023-10-15 17:09:33,708][52833] Updated weights for policy 0, policy_version 59250 (0.0008) -[2023-10-15 17:09:34,080][52833] Updated weights for policy 0, policy_version 59260 (0.0008) -[2023-10-15 17:09:36,246][52866] Updated weights for policy 1, policy_version 59460 (0.0011) -[2023-10-15 17:09:36,617][52866] Updated weights for policy 1, policy_version 59470 (0.0010) -[2023-10-15 17:09:36,983][52866] Updated weights for policy 1, policy_version 59480 (0.0008) -[2023-10-15 17:09:37,902][52833] Updated weights for policy 0, policy_version 59270 (0.0010) -[2023-10-15 17:09:38,268][52833] Updated weights for policy 0, policy_version 59280 (0.0010) -[2023-10-15 17:09:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 121602048. Throughput: 0: 1796.7, 1: 1805.4. Samples: 30409710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:38,442][51532] Avg episode reward: [(0, '56.010'), (1, '58.980')] -[2023-10-15 17:09:38,647][52833] Updated weights for policy 0, policy_version 59290 (0.0009) -[2023-10-15 17:09:40,723][52866] Updated weights for policy 1, policy_version 59490 (0.0007) -[2023-10-15 17:09:41,093][52866] Updated weights for policy 1, policy_version 59500 (0.0008) -[2023-10-15 17:09:41,468][52866] Updated weights for policy 1, policy_version 59510 (0.0009) -[2023-10-15 17:09:41,826][52866] Updated weights for policy 1, policy_version 59520 (0.0011) -[2023-10-15 17:09:42,414][52833] Updated weights for policy 0, policy_version 59300 (0.0009) -[2023-10-15 17:09:42,778][52833] Updated weights for policy 0, policy_version 59310 (0.0007) -[2023-10-15 17:09:43,144][52833] Updated weights for policy 0, policy_version 59320 (0.0008) -[2023-10-15 17:09:43,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 121700352. Throughput: 0: 1809.3, 1: 1799.2. Samples: 30431264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:43,442][51532] Avg episode reward: [(0, '55.200'), (1, '60.070')] -[2023-10-15 17:09:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000059520_60948480.pth... -[2023-10-15 17:09:43,453][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000059328_60751872.pth... -[2023-10-15 17:09:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000057632_59015168.pth -[2023-10-15 17:09:43,493][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000057824_59211776.pth -[2023-10-15 17:09:45,654][52866] Updated weights for policy 1, policy_version 59530 (0.0009) -[2023-10-15 17:09:46,017][52866] Updated weights for policy 1, policy_version 59540 (0.0009) -[2023-10-15 17:09:46,380][52866] Updated weights for policy 1, policy_version 59550 (0.0008) -[2023-10-15 17:09:46,802][52833] Updated weights for policy 0, policy_version 59330 (0.0009) -[2023-10-15 17:09:47,163][52833] Updated weights for policy 0, policy_version 59340 (0.0007) -[2023-10-15 17:09:47,542][52833] Updated weights for policy 0, policy_version 59350 (0.0007) -[2023-10-15 17:09:47,905][52833] Updated weights for policy 0, policy_version 59360 (0.0008) -[2023-10-15 17:09:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 121765888. Throughput: 0: 1794.8, 1: 1813.2. Samples: 30442464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:48,441][51532] Avg episode reward: [(0, '54.260'), (1, '61.360')] -[2023-10-15 17:09:48,442][52518] Saving new best policy, reward=61.360! -[2023-10-15 17:09:50,242][52866] Updated weights for policy 1, policy_version 59560 (0.0009) -[2023-10-15 17:09:50,616][52866] Updated weights for policy 1, policy_version 59570 (0.0011) -[2023-10-15 17:09:50,986][52866] Updated weights for policy 1, policy_version 59580 (0.0008) -[2023-10-15 17:09:51,671][52833] Updated weights for policy 0, policy_version 59370 (0.0008) -[2023-10-15 17:09:52,044][52833] Updated weights for policy 0, policy_version 59380 (0.0007) -[2023-10-15 17:09:52,418][52833] Updated weights for policy 0, policy_version 59390 (0.0007) -[2023-10-15 17:09:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 121831424. Throughput: 0: 1802.7, 1: 1798.7. Samples: 30463326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:09:53,441][51532] Avg episode reward: [(0, '54.520'), (1, '64.510')] -[2023-10-15 17:09:53,442][52518] Saving new best policy, reward=64.510! -[2023-10-15 17:09:54,534][52866] Updated weights for policy 1, policy_version 59590 (0.0010) -[2023-10-15 17:09:54,892][52866] Updated weights for policy 1, policy_version 59600 (0.0009) -[2023-10-15 17:09:55,258][52866] Updated weights for policy 1, policy_version 59610 (0.0008) -[2023-10-15 17:09:56,139][52833] Updated weights for policy 0, policy_version 59400 (0.0008) -[2023-10-15 17:09:56,515][52833] Updated weights for policy 0, policy_version 59410 (0.0010) -[2023-10-15 17:09:56,897][52833] Updated weights for policy 0, policy_version 59420 (0.0010) -[2023-10-15 17:09:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 121896960. Throughput: 0: 1791.3, 1: 1796.4. Samples: 30485420. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:09:58,442][51532] Avg episode reward: [(0, '51.390'), (1, '62.390')] -[2023-10-15 17:09:58,922][52866] Updated weights for policy 1, policy_version 59620 (0.0008) -[2023-10-15 17:09:59,283][52866] Updated weights for policy 1, policy_version 59630 (0.0008) -[2023-10-15 17:09:59,648][52866] Updated weights for policy 1, policy_version 59640 (0.0010) -[2023-10-15 17:10:00,568][52833] Updated weights for policy 0, policy_version 59430 (0.0009) -[2023-10-15 17:10:00,932][52833] Updated weights for policy 0, policy_version 59440 (0.0007) -[2023-10-15 17:10:01,300][52833] Updated weights for policy 0, policy_version 59450 (0.0007) -[2023-10-15 17:10:03,344][52866] Updated weights for policy 1, policy_version 59650 (0.0010) -[2023-10-15 17:10:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 121962496. Throughput: 0: 1804.6, 1: 1800.6. Samples: 30496406. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:10:03,441][51532] Avg episode reward: [(0, '49.900'), (1, '64.210')] -[2023-10-15 17:10:03,706][52866] Updated weights for policy 1, policy_version 59660 (0.0008) -[2023-10-15 17:10:04,076][52866] Updated weights for policy 1, policy_version 59670 (0.0008) -[2023-10-15 17:10:04,432][52866] Updated weights for policy 1, policy_version 59680 (0.0009) -[2023-10-15 17:10:05,161][52833] Updated weights for policy 0, policy_version 59460 (0.0009) -[2023-10-15 17:10:05,530][52833] Updated weights for policy 0, policy_version 59470 (0.0007) -[2023-10-15 17:10:05,892][52833] Updated weights for policy 0, policy_version 59480 (0.0007) -[2023-10-15 17:10:08,107][52866] Updated weights for policy 1, policy_version 59690 (0.0007) -[2023-10-15 17:10:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 122028032. Throughput: 0: 1791.0, 1: 1808.3. Samples: 30518142. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:10:08,441][51532] Avg episode reward: [(0, '50.410'), (1, '66.270')] -[2023-10-15 17:10:08,477][52866] Updated weights for policy 1, policy_version 59700 (0.0007) -[2023-10-15 17:10:08,844][52866] Updated weights for policy 1, policy_version 59710 (0.0008) -[2023-10-15 17:10:08,916][52518] Saving new best policy, reward=66.270! -[2023-10-15 17:10:09,637][52833] Updated weights for policy 0, policy_version 59490 (0.0007) -[2023-10-15 17:10:10,016][52833] Updated weights for policy 0, policy_version 59500 (0.0010) -[2023-10-15 17:10:10,387][52833] Updated weights for policy 0, policy_version 59510 (0.0007) -[2023-10-15 17:10:10,764][52833] Updated weights for policy 0, policy_version 59520 (0.0007) -[2023-10-15 17:10:12,470][52866] Updated weights for policy 1, policy_version 59720 (0.0007) -[2023-10-15 17:10:12,845][52866] Updated weights for policy 1, policy_version 59730 (0.0007) -[2023-10-15 17:10:13,213][52866] Updated weights for policy 1, policy_version 59740 (0.0007) -[2023-10-15 17:10:13,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 122126336. Throughput: 0: 1782.7, 1: 1818.4. Samples: 30539650. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:10:13,442][51532] Avg episode reward: [(0, '48.510'), (1, '65.680')] -[2023-10-15 17:10:14,656][52833] Updated weights for policy 0, policy_version 59530 (0.0008) -[2023-10-15 17:10:15,040][52833] Updated weights for policy 0, policy_version 59540 (0.0008) -[2023-10-15 17:10:15,413][52833] Updated weights for policy 0, policy_version 59550 (0.0008) -[2023-10-15 17:10:16,911][52866] Updated weights for policy 1, policy_version 59750 (0.0008) -[2023-10-15 17:10:17,276][52866] Updated weights for policy 1, policy_version 59760 (0.0007) -[2023-10-15 17:10:17,641][52866] Updated weights for policy 1, policy_version 59770 (0.0008) -[2023-10-15 17:10:18,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 122191872. Throughput: 0: 1779.4, 1: 1811.6. Samples: 30550110. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:10:18,442][51532] Avg episode reward: [(0, '49.920'), (1, '67.930')] -[2023-10-15 17:10:18,443][52518] Saving new best policy, reward=67.930! -[2023-10-15 17:10:19,187][52833] Updated weights for policy 0, policy_version 59560 (0.0009) -[2023-10-15 17:10:19,550][52833] Updated weights for policy 0, policy_version 59570 (0.0007) -[2023-10-15 17:10:19,917][52833] Updated weights for policy 0, policy_version 59580 (0.0008) -[2023-10-15 17:10:21,495][52866] Updated weights for policy 1, policy_version 59780 (0.0010) -[2023-10-15 17:10:21,869][52866] Updated weights for policy 1, policy_version 59790 (0.0007) -[2023-10-15 17:10:22,239][52866] Updated weights for policy 1, policy_version 59800 (0.0007) -[2023-10-15 17:10:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 122257408. Throughput: 0: 1785.6, 1: 1816.0. Samples: 30571782. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:10:23,442][51532] Avg episode reward: [(0, '46.280'), (1, '67.080')] -[2023-10-15 17:10:23,574][52833] Updated weights for policy 0, policy_version 59590 (0.0011) -[2023-10-15 17:10:23,945][52833] Updated weights for policy 0, policy_version 59600 (0.0010) -[2023-10-15 17:10:24,317][52833] Updated weights for policy 0, policy_version 59610 (0.0009) -[2023-10-15 17:10:25,869][52866] Updated weights for policy 1, policy_version 59810 (0.0010) -[2023-10-15 17:10:26,236][52866] Updated weights for policy 1, policy_version 59820 (0.0008) -[2023-10-15 17:10:26,603][52866] Updated weights for policy 1, policy_version 59830 (0.0008) -[2023-10-15 17:10:26,960][52866] Updated weights for policy 1, policy_version 59840 (0.0008) -[2023-10-15 17:10:28,130][52833] Updated weights for policy 0, policy_version 59620 (0.0010) -[2023-10-15 17:10:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 122322944. Throughput: 0: 1804.7, 1: 1808.1. Samples: 30593842. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:10:28,442][51532] Avg episode reward: [(0, '44.840'), (1, '64.050')] -[2023-10-15 17:10:28,495][52833] Updated weights for policy 0, policy_version 59630 (0.0010) -[2023-10-15 17:10:28,855][52833] Updated weights for policy 0, policy_version 59640 (0.0009) -[2023-10-15 17:10:30,866][52866] Updated weights for policy 1, policy_version 59850 (0.0008) -[2023-10-15 17:10:31,235][52866] Updated weights for policy 1, policy_version 59860 (0.0009) -[2023-10-15 17:10:31,596][52866] Updated weights for policy 1, policy_version 59870 (0.0011) -[2023-10-15 17:10:32,674][52833] Updated weights for policy 0, policy_version 59650 (0.0010) -[2023-10-15 17:10:33,054][52833] Updated weights for policy 0, policy_version 59660 (0.0008) -[2023-10-15 17:10:33,426][52833] Updated weights for policy 0, policy_version 59670 (0.0009) -[2023-10-15 17:10:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 122388480. Throughput: 0: 1786.9, 1: 1816.0. Samples: 30604594. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:10:33,441][51532] Avg episode reward: [(0, '45.270'), (1, '65.570')] -[2023-10-15 17:10:33,793][52833] Updated weights for policy 0, policy_version 59680 (0.0007) -[2023-10-15 17:10:35,241][52866] Updated weights for policy 1, policy_version 59880 (0.0008) -[2023-10-15 17:10:35,608][52866] Updated weights for policy 1, policy_version 59890 (0.0007) -[2023-10-15 17:10:35,976][52866] Updated weights for policy 1, policy_version 59900 (0.0008) -[2023-10-15 17:10:37,654][52833] Updated weights for policy 0, policy_version 59690 (0.0008) -[2023-10-15 17:10:38,025][52833] Updated weights for policy 0, policy_version 59700 (0.0007) -[2023-10-15 17:10:38,389][52833] Updated weights for policy 0, policy_version 59710 (0.0008) -[2023-10-15 17:10:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 122454016. Throughput: 0: 1803.0, 1: 1818.4. Samples: 30626290. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-15 17:10:38,442][51532] Avg episode reward: [(0, '44.560'), (1, '64.620')] -[2023-10-15 17:10:39,585][52866] Updated weights for policy 1, policy_version 59910 (0.0009) -[2023-10-15 17:10:39,956][52866] Updated weights for policy 1, policy_version 59920 (0.0007) -[2023-10-15 17:10:40,319][52866] Updated weights for policy 1, policy_version 59930 (0.0008) -[2023-10-15 17:10:41,936][52833] Updated weights for policy 0, policy_version 59720 (0.0008) -[2023-10-15 17:10:42,301][52833] Updated weights for policy 0, policy_version 59730 (0.0008) -[2023-10-15 17:10:42,672][52833] Updated weights for policy 0, policy_version 59740 (0.0007) -[2023-10-15 17:10:43,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 122552320. Throughput: 0: 1789.3, 1: 1814.5. Samples: 30647590. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) -[2023-10-15 17:10:43,442][51532] Avg episode reward: [(0, '45.690'), (1, '62.850')] -[2023-10-15 17:10:43,920][52866] Updated weights for policy 1, policy_version 59940 (0.0009) -[2023-10-15 17:10:44,287][52866] Updated weights for policy 1, policy_version 59950 (0.0009) -[2023-10-15 17:10:44,645][52866] Updated weights for policy 1, policy_version 59960 (0.0008) -[2023-10-15 17:10:46,551][52833] Updated weights for policy 0, policy_version 59750 (0.0010) -[2023-10-15 17:10:46,925][52833] Updated weights for policy 0, policy_version 59760 (0.0008) -[2023-10-15 17:10:47,301][52833] Updated weights for policy 0, policy_version 59770 (0.0007) -[2023-10-15 17:10:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 122617856. Throughput: 0: 1797.8, 1: 1808.7. Samples: 30658696. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) -[2023-10-15 17:10:48,441][51532] Avg episode reward: [(0, '47.300'), (1, '63.740')] -[2023-10-15 17:10:48,502][52866] Updated weights for policy 1, policy_version 59970 (0.0009) -[2023-10-15 17:10:48,862][52866] Updated weights for policy 1, policy_version 59980 (0.0008) -[2023-10-15 17:10:49,240][52866] Updated weights for policy 1, policy_version 59990 (0.0010) -[2023-10-15 17:10:49,601][52866] Updated weights for policy 1, policy_version 60000 (0.0008) -[2023-10-15 17:10:51,055][52833] Updated weights for policy 0, policy_version 59780 (0.0009) -[2023-10-15 17:10:51,417][52833] Updated weights for policy 0, policy_version 59790 (0.0011) -[2023-10-15 17:10:51,792][52833] Updated weights for policy 0, policy_version 59800 (0.0009) -[2023-10-15 17:10:53,253][52866] Updated weights for policy 1, policy_version 60010 (0.0007) -[2023-10-15 17:10:53,441][51532] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 122683392. Throughput: 0: 1790.2, 1: 1803.2. Samples: 30679848. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) -[2023-10-15 17:10:53,442][51532] Avg episode reward: [(0, '51.060'), (1, '61.490')] -[2023-10-15 17:10:53,623][52866] Updated weights for policy 1, policy_version 60020 (0.0007) -[2023-10-15 17:10:53,998][52866] Updated weights for policy 1, policy_version 60030 (0.0009) -[2023-10-15 17:10:55,439][52833] Updated weights for policy 0, policy_version 59810 (0.0010) -[2023-10-15 17:10:55,809][52833] Updated weights for policy 0, policy_version 59820 (0.0010) -[2023-10-15 17:10:56,180][52833] Updated weights for policy 0, policy_version 59830 (0.0007) -[2023-10-15 17:10:56,550][52833] Updated weights for policy 0, policy_version 59840 (0.0008) -[2023-10-15 17:10:57,654][52866] Updated weights for policy 1, policy_version 60040 (0.0010) -[2023-10-15 17:10:58,015][52866] Updated weights for policy 1, policy_version 60050 (0.0010) -[2023-10-15 17:10:58,384][52866] Updated weights for policy 1, policy_version 60060 (0.0010) -[2023-10-15 17:10:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 122748928. Throughput: 0: 1788.6, 1: 1809.2. Samples: 30701550. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) -[2023-10-15 17:10:58,442][51532] Avg episode reward: [(0, '53.080'), (1, '60.490')] -[2023-10-15 17:11:00,485][52833] Updated weights for policy 0, policy_version 59850 (0.0008) -[2023-10-15 17:11:00,850][52833] Updated weights for policy 0, policy_version 59860 (0.0007) -[2023-10-15 17:11:01,209][52833] Updated weights for policy 0, policy_version 59870 (0.0009) -[2023-10-15 17:11:02,217][52866] Updated weights for policy 1, policy_version 60070 (0.0011) -[2023-10-15 17:11:02,587][52866] Updated weights for policy 1, policy_version 60080 (0.0008) -[2023-10-15 17:11:02,949][52866] Updated weights for policy 1, policy_version 60090 (0.0008) -[2023-10-15 17:11:03,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 122847232. Throughput: 0: 1801.5, 1: 1804.6. Samples: 30712384. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) -[2023-10-15 17:11:03,442][51532] Avg episode reward: [(0, '51.800'), (1, '61.370')] -[2023-10-15 17:11:04,872][52833] Updated weights for policy 0, policy_version 59880 (0.0010) -[2023-10-15 17:11:05,238][52833] Updated weights for policy 0, policy_version 59890 (0.0009) -[2023-10-15 17:11:05,604][52833] Updated weights for policy 0, policy_version 59900 (0.0010) -[2023-10-15 17:11:06,720][52866] Updated weights for policy 1, policy_version 60100 (0.0008) -[2023-10-15 17:11:07,077][52866] Updated weights for policy 1, policy_version 60110 (0.0007) -[2023-10-15 17:11:07,441][52866] Updated weights for policy 1, policy_version 60120 (0.0007) -[2023-10-15 17:11:08,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 122912768. Throughput: 0: 1786.7, 1: 1817.4. Samples: 30733966. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) -[2023-10-15 17:11:08,441][51532] Avg episode reward: [(0, '54.650'), (1, '63.310')] -[2023-10-15 17:11:09,348][52833] Updated weights for policy 0, policy_version 59910 (0.0009) -[2023-10-15 17:11:09,720][52833] Updated weights for policy 0, policy_version 59920 (0.0008) -[2023-10-15 17:11:10,090][52833] Updated weights for policy 0, policy_version 59930 (0.0008) -[2023-10-15 17:11:11,022][52866] Updated weights for policy 1, policy_version 60130 (0.0008) -[2023-10-15 17:11:11,384][52866] Updated weights for policy 1, policy_version 60140 (0.0009) -[2023-10-15 17:11:11,752][52866] Updated weights for policy 1, policy_version 60150 (0.0010) -[2023-10-15 17:11:12,126][52866] Updated weights for policy 1, policy_version 60160 (0.0007) -[2023-10-15 17:11:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 122978304. Throughput: 0: 1781.3, 1: 1815.9. Samples: 30755716. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) -[2023-10-15 17:11:13,442][51532] Avg episode reward: [(0, '54.990'), (1, '63.590')] -[2023-10-15 17:11:13,811][52833] Updated weights for policy 0, policy_version 59940 (0.0009) -[2023-10-15 17:11:14,185][52833] Updated weights for policy 0, policy_version 59950 (0.0010) -[2023-10-15 17:11:14,543][52833] Updated weights for policy 0, policy_version 59960 (0.0008) -[2023-10-15 17:11:15,985][52866] Updated weights for policy 1, policy_version 60170 (0.0008) -[2023-10-15 17:11:16,347][52866] Updated weights for policy 1, policy_version 60180 (0.0007) -[2023-10-15 17:11:16,709][52866] Updated weights for policy 1, policy_version 60190 (0.0009) -[2023-10-15 17:11:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 123043840. Throughput: 0: 1783.0, 1: 1817.2. Samples: 30766604. Policy #0 lag: (min: 31.0, avg: 46.3, max: 63.0) -[2023-10-15 17:11:18,441][51532] Avg episode reward: [(0, '57.940'), (1, '65.000')] -[2023-10-15 17:11:18,449][52833] Updated weights for policy 0, policy_version 59970 (0.0009) -[2023-10-15 17:11:18,819][52833] Updated weights for policy 0, policy_version 59980 (0.0007) -[2023-10-15 17:11:19,192][52833] Updated weights for policy 0, policy_version 59990 (0.0008) -[2023-10-15 17:11:19,558][52833] Updated weights for policy 0, policy_version 60000 (0.0008) -[2023-10-15 17:11:20,416][52866] Updated weights for policy 1, policy_version 60200 (0.0008) -[2023-10-15 17:11:20,791][52866] Updated weights for policy 1, policy_version 60210 (0.0007) -[2023-10-15 17:11:21,154][52866] Updated weights for policy 1, policy_version 60220 (0.0007) -[2023-10-15 17:11:23,174][52833] Updated weights for policy 0, policy_version 60010 (0.0011) -[2023-10-15 17:11:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 123109376. Throughput: 0: 1782.4, 1: 1813.8. Samples: 30788118. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 17:11:23,441][51532] Avg episode reward: [(0, '56.480'), (1, '62.710')] -[2023-10-15 17:11:23,543][52833] Updated weights for policy 0, policy_version 60020 (0.0011) -[2023-10-15 17:11:23,919][52833] Updated weights for policy 0, policy_version 60030 (0.0011) -[2023-10-15 17:11:24,833][52866] Updated weights for policy 1, policy_version 60230 (0.0010) -[2023-10-15 17:11:25,189][52866] Updated weights for policy 1, policy_version 60240 (0.0009) -[2023-10-15 17:11:25,558][52866] Updated weights for policy 1, policy_version 60250 (0.0009) -[2023-10-15 17:11:27,751][52833] Updated weights for policy 0, policy_version 60040 (0.0010) -[2023-10-15 17:11:28,127][52833] Updated weights for policy 0, policy_version 60050 (0.0008) -[2023-10-15 17:11:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 123174912. Throughput: 0: 1799.1, 1: 1811.3. Samples: 30810056. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 17:11:28,442][51532] Avg episode reward: [(0, '57.670'), (1, '61.470')] -[2023-10-15 17:11:28,501][52833] Updated weights for policy 0, policy_version 60060 (0.0011) -[2023-10-15 17:11:29,533][52866] Updated weights for policy 1, policy_version 60260 (0.0009) -[2023-10-15 17:11:29,892][52866] Updated weights for policy 1, policy_version 60270 (0.0008) -[2023-10-15 17:11:30,259][52866] Updated weights for policy 1, policy_version 60280 (0.0009) -[2023-10-15 17:11:32,234][52833] Updated weights for policy 0, policy_version 60070 (0.0009) -[2023-10-15 17:11:32,602][52833] Updated weights for policy 0, policy_version 60080 (0.0007) -[2023-10-15 17:11:32,977][52833] Updated weights for policy 0, policy_version 60090 (0.0008) -[2023-10-15 17:11:33,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 123273216. Throughput: 0: 1778.2, 1: 1812.8. Samples: 30820294. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 17:11:33,442][51532] Avg episode reward: [(0, '56.640'), (1, '59.270')] -[2023-10-15 17:11:34,028][52866] Updated weights for policy 1, policy_version 60290 (0.0007) -[2023-10-15 17:11:34,401][52866] Updated weights for policy 1, policy_version 60300 (0.0008) -[2023-10-15 17:11:34,766][52866] Updated weights for policy 1, policy_version 60310 (0.0008) -[2023-10-15 17:11:35,126][52866] Updated weights for policy 1, policy_version 60320 (0.0009) -[2023-10-15 17:11:36,698][52833] Updated weights for policy 0, policy_version 60100 (0.0009) -[2023-10-15 17:11:37,065][52833] Updated weights for policy 0, policy_version 60110 (0.0008) -[2023-10-15 17:11:37,443][52833] Updated weights for policy 0, policy_version 60120 (0.0009) -[2023-10-15 17:11:38,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 123338752. Throughput: 0: 1795.9, 1: 1816.1. Samples: 30842384. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 17:11:38,441][51532] Avg episode reward: [(0, '54.790'), (1, '58.870')] -[2023-10-15 17:11:38,830][52866] Updated weights for policy 1, policy_version 60330 (0.0009) -[2023-10-15 17:11:39,193][52866] Updated weights for policy 1, policy_version 60340 (0.0009) -[2023-10-15 17:11:39,566][52866] Updated weights for policy 1, policy_version 60350 (0.0007) -[2023-10-15 17:11:41,093][52833] Updated weights for policy 0, policy_version 60130 (0.0009) -[2023-10-15 17:11:41,459][52833] Updated weights for policy 0, policy_version 60140 (0.0009) -[2023-10-15 17:11:41,825][52833] Updated weights for policy 0, policy_version 60150 (0.0009) -[2023-10-15 17:11:42,190][52833] Updated weights for policy 0, policy_version 60160 (0.0007) -[2023-10-15 17:11:43,176][52866] Updated weights for policy 1, policy_version 60360 (0.0010) -[2023-10-15 17:11:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 123404288. Throughput: 0: 1781.6, 1: 1827.6. Samples: 30863964. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 17:11:43,442][51532] Avg episode reward: [(0, '56.040'), (1, '61.580')] -[2023-10-15 17:11:43,453][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000060160_61603840.pth... -[2023-10-15 17:11:43,488][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000058464_59867136.pth -[2023-10-15 17:11:43,535][52866] Updated weights for policy 1, policy_version 60370 (0.0009) -[2023-10-15 17:11:43,895][52866] Updated weights for policy 1, policy_version 60380 (0.0009) -[2023-10-15 17:11:44,040][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000060384_61833216.pth... -[2023-10-15 17:11:44,069][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000058688_60096512.pth -[2023-10-15 17:11:46,070][52833] Updated weights for policy 0, policy_version 60170 (0.0009) -[2023-10-15 17:11:46,428][52833] Updated weights for policy 0, policy_version 60180 (0.0009) -[2023-10-15 17:11:46,800][52833] Updated weights for policy 0, policy_version 60190 (0.0008) -[2023-10-15 17:11:47,719][52866] Updated weights for policy 1, policy_version 60390 (0.0009) -[2023-10-15 17:11:48,082][52866] Updated weights for policy 1, policy_version 60400 (0.0009) -[2023-10-15 17:11:48,440][52866] Updated weights for policy 1, policy_version 60410 (0.0010) -[2023-10-15 17:11:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 123469824. Throughput: 0: 1800.6, 1: 1809.5. Samples: 30874838. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 17:11:48,441][51532] Avg episode reward: [(0, '56.650'), (1, '61.110')] -[2023-10-15 17:11:50,458][52833] Updated weights for policy 0, policy_version 60200 (0.0009) -[2023-10-15 17:11:50,835][52833] Updated weights for policy 0, policy_version 60210 (0.0008) -[2023-10-15 17:11:51,206][52833] Updated weights for policy 0, policy_version 60220 (0.0008) -[2023-10-15 17:11:52,161][52866] Updated weights for policy 1, policy_version 60420 (0.0008) -[2023-10-15 17:11:52,528][52866] Updated weights for policy 1, policy_version 60430 (0.0007) -[2023-10-15 17:11:52,901][52866] Updated weights for policy 1, policy_version 60440 (0.0009) -[2023-10-15 17:11:53,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 123568128. Throughput: 0: 1786.4, 1: 1816.6. Samples: 30896102. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 17:11:53,442][51532] Avg episode reward: [(0, '57.980'), (1, '59.200')] -[2023-10-15 17:11:55,000][52833] Updated weights for policy 0, policy_version 60230 (0.0009) -[2023-10-15 17:11:55,369][52833] Updated weights for policy 0, policy_version 60240 (0.0009) -[2023-10-15 17:11:55,742][52833] Updated weights for policy 0, policy_version 60250 (0.0007) -[2023-10-15 17:11:56,475][52866] Updated weights for policy 1, policy_version 60450 (0.0008) -[2023-10-15 17:11:56,848][52866] Updated weights for policy 1, policy_version 60460 (0.0010) -[2023-10-15 17:11:57,207][52866] Updated weights for policy 1, policy_version 60470 (0.0009) -[2023-10-15 17:11:57,578][52866] Updated weights for policy 1, policy_version 60480 (0.0009) -[2023-10-15 17:11:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.2). Total num frames: 123633664. Throughput: 0: 1789.1, 1: 1805.1. Samples: 30917456. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) -[2023-10-15 17:11:58,441][51532] Avg episode reward: [(0, '56.940'), (1, '57.840')] -[2023-10-15 17:11:59,539][52833] Updated weights for policy 0, policy_version 60260 (0.0009) -[2023-10-15 17:11:59,918][52833] Updated weights for policy 0, policy_version 60270 (0.0011) -[2023-10-15 17:12:00,286][52833] Updated weights for policy 0, policy_version 60280 (0.0008) -[2023-10-15 17:12:01,497][52866] Updated weights for policy 1, policy_version 60490 (0.0008) -[2023-10-15 17:12:01,875][52866] Updated weights for policy 1, policy_version 60500 (0.0009) -[2023-10-15 17:12:02,246][52866] Updated weights for policy 1, policy_version 60510 (0.0008) -[2023-10-15 17:12:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 123699200. Throughput: 0: 1785.9, 1: 1811.3. Samples: 30928476. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:12:03,442][51532] Avg episode reward: [(0, '58.340'), (1, '54.990')] -[2023-10-15 17:12:03,955][52833] Updated weights for policy 0, policy_version 60290 (0.0009) -[2023-10-15 17:12:04,319][52833] Updated weights for policy 0, policy_version 60300 (0.0007) -[2023-10-15 17:12:04,695][52833] Updated weights for policy 0, policy_version 60310 (0.0009) -[2023-10-15 17:12:05,063][52833] Updated weights for policy 0, policy_version 60320 (0.0007) -[2023-10-15 17:12:05,823][52866] Updated weights for policy 1, policy_version 60520 (0.0007) -[2023-10-15 17:12:06,193][52866] Updated weights for policy 1, policy_version 60530 (0.0007) -[2023-10-15 17:12:06,559][52866] Updated weights for policy 1, policy_version 60540 (0.0008) -[2023-10-15 17:12:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 123764736. Throughput: 0: 1794.7, 1: 1799.8. Samples: 30949872. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:12:08,442][51532] Avg episode reward: [(0, '58.090'), (1, '55.780')] -[2023-10-15 17:12:08,722][52833] Updated weights for policy 0, policy_version 60330 (0.0010) -[2023-10-15 17:12:09,091][52833] Updated weights for policy 0, policy_version 60340 (0.0010) -[2023-10-15 17:12:09,461][52833] Updated weights for policy 0, policy_version 60350 (0.0008) -[2023-10-15 17:12:10,148][52866] Updated weights for policy 1, policy_version 60550 (0.0010) -[2023-10-15 17:12:10,510][52866] Updated weights for policy 1, policy_version 60560 (0.0010) -[2023-10-15 17:12:10,881][52866] Updated weights for policy 1, policy_version 60570 (0.0008) -[2023-10-15 17:12:13,292][52833] Updated weights for policy 0, policy_version 60360 (0.0007) -[2023-10-15 17:12:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 123830272. Throughput: 0: 1810.3, 1: 1800.5. Samples: 30972542. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:12:13,442][51532] Avg episode reward: [(0, '57.630'), (1, '55.490')] -[2023-10-15 17:12:13,666][52833] Updated weights for policy 0, policy_version 60370 (0.0007) -[2023-10-15 17:12:14,033][52833] Updated weights for policy 0, policy_version 60380 (0.0009) -[2023-10-15 17:12:14,621][52866] Updated weights for policy 1, policy_version 60580 (0.0009) -[2023-10-15 17:12:14,992][52866] Updated weights for policy 1, policy_version 60590 (0.0009) -[2023-10-15 17:12:15,355][52866] Updated weights for policy 1, policy_version 60600 (0.0008) -[2023-10-15 17:12:17,665][52833] Updated weights for policy 0, policy_version 60390 (0.0008) -[2023-10-15 17:12:18,041][52833] Updated weights for policy 0, policy_version 60400 (0.0009) -[2023-10-15 17:12:18,405][52833] Updated weights for policy 0, policy_version 60410 (0.0008) -[2023-10-15 17:12:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 123895808. Throughput: 0: 1801.2, 1: 1799.9. Samples: 30982342. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:12:18,442][51532] Avg episode reward: [(0, '55.710'), (1, '56.550')] -[2023-10-15 17:12:19,170][52866] Updated weights for policy 1, policy_version 60610 (0.0008) -[2023-10-15 17:12:19,537][52866] Updated weights for policy 1, policy_version 60620 (0.0010) -[2023-10-15 17:12:19,908][52866] Updated weights for policy 1, policy_version 60630 (0.0011) -[2023-10-15 17:12:20,269][52866] Updated weights for policy 1, policy_version 60640 (0.0011) -[2023-10-15 17:12:22,136][52833] Updated weights for policy 0, policy_version 60420 (0.0007) -[2023-10-15 17:12:22,501][52833] Updated weights for policy 0, policy_version 60430 (0.0008) -[2023-10-15 17:12:22,869][52833] Updated weights for policy 0, policy_version 60440 (0.0007) -[2023-10-15 17:12:23,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 123994112. Throughput: 0: 1814.4, 1: 1793.2. Samples: 31004726. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:12:23,442][51532] Avg episode reward: [(0, '53.350'), (1, '57.630')] -[2023-10-15 17:12:23,911][52866] Updated weights for policy 1, policy_version 60650 (0.0008) -[2023-10-15 17:12:24,282][52866] Updated weights for policy 1, policy_version 60660 (0.0007) -[2023-10-15 17:12:24,655][52866] Updated weights for policy 1, policy_version 60670 (0.0009) -[2023-10-15 17:12:26,583][52833] Updated weights for policy 0, policy_version 60450 (0.0007) -[2023-10-15 17:12:26,957][52833] Updated weights for policy 0, policy_version 60460 (0.0008) -[2023-10-15 17:12:27,330][52833] Updated weights for policy 0, policy_version 60470 (0.0007) -[2023-10-15 17:12:27,688][52833] Updated weights for policy 0, policy_version 60480 (0.0008) -[2023-10-15 17:12:28,385][52866] Updated weights for policy 1, policy_version 60680 (0.0010) -[2023-10-15 17:12:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 124059648. Throughput: 0: 1798.1, 1: 1800.9. Samples: 31025920. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:12:28,441][51532] Avg episode reward: [(0, '55.940'), (1, '57.730')] -[2023-10-15 17:12:28,749][52866] Updated weights for policy 1, policy_version 60690 (0.0012) -[2023-10-15 17:12:29,123][52866] Updated weights for policy 1, policy_version 60700 (0.0011) -[2023-10-15 17:12:31,564][52833] Updated weights for policy 0, policy_version 60490 (0.0010) -[2023-10-15 17:12:31,934][52833] Updated weights for policy 0, policy_version 60500 (0.0009) -[2023-10-15 17:12:32,308][52833] Updated weights for policy 0, policy_version 60510 (0.0008) -[2023-10-15 17:12:32,960][52866] Updated weights for policy 1, policy_version 60710 (0.0010) -[2023-10-15 17:12:33,326][52866] Updated weights for policy 1, policy_version 60720 (0.0010) -[2023-10-15 17:12:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 124125184. Throughput: 0: 1805.6, 1: 1798.6. Samples: 31037024. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:12:33,441][51532] Avg episode reward: [(0, '53.430'), (1, '61.170')] -[2023-10-15 17:12:33,695][52866] Updated weights for policy 1, policy_version 60730 (0.0007) -[2023-10-15 17:12:36,057][52833] Updated weights for policy 0, policy_version 60520 (0.0007) -[2023-10-15 17:12:36,438][52833] Updated weights for policy 0, policy_version 60530 (0.0008) -[2023-10-15 17:12:36,808][52833] Updated weights for policy 0, policy_version 60540 (0.0009) -[2023-10-15 17:12:37,428][52866] Updated weights for policy 1, policy_version 60740 (0.0008) -[2023-10-15 17:12:37,787][52866] Updated weights for policy 1, policy_version 60750 (0.0011) -[2023-10-15 17:12:38,159][52866] Updated weights for policy 1, policy_version 60760 (0.0010) -[2023-10-15 17:12:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 124190720. Throughput: 0: 1801.7, 1: 1803.6. Samples: 31058344. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:12:38,442][51532] Avg episode reward: [(0, '51.670'), (1, '59.570')] -[2023-10-15 17:12:40,533][52833] Updated weights for policy 0, policy_version 60550 (0.0008) -[2023-10-15 17:12:40,891][52833] Updated weights for policy 0, policy_version 60560 (0.0009) -[2023-10-15 17:12:41,258][52833] Updated weights for policy 0, policy_version 60570 (0.0010) -[2023-10-15 17:12:41,828][52866] Updated weights for policy 1, policy_version 60770 (0.0008) -[2023-10-15 17:12:42,193][52866] Updated weights for policy 1, policy_version 60780 (0.0008) -[2023-10-15 17:12:42,573][52866] Updated weights for policy 1, policy_version 60790 (0.0007) -[2023-10-15 17:12:42,941][52866] Updated weights for policy 1, policy_version 60800 (0.0011) -[2023-10-15 17:12:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 124289024. Throughput: 0: 1794.9, 1: 1799.9. Samples: 31079224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 17:12:43,442][51532] Avg episode reward: [(0, '50.330'), (1, '58.060')] -[2023-10-15 17:12:45,018][52833] Updated weights for policy 0, policy_version 60580 (0.0011) -[2023-10-15 17:12:45,397][52833] Updated weights for policy 0, policy_version 60590 (0.0008) -[2023-10-15 17:12:45,766][52833] Updated weights for policy 0, policy_version 60600 (0.0008) -[2023-10-15 17:12:46,691][52866] Updated weights for policy 1, policy_version 60810 (0.0010) -[2023-10-15 17:12:47,066][52866] Updated weights for policy 1, policy_version 60820 (0.0011) -[2023-10-15 17:12:47,437][52866] Updated weights for policy 1, policy_version 60830 (0.0008) -[2023-10-15 17:12:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 124354560. Throughput: 0: 1802.8, 1: 1805.5. Samples: 31090848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 17:12:48,441][51532] Avg episode reward: [(0, '52.400'), (1, '58.350')] -[2023-10-15 17:12:49,506][52833] Updated weights for policy 0, policy_version 60610 (0.0008) -[2023-10-15 17:12:49,878][52833] Updated weights for policy 0, policy_version 60620 (0.0008) -[2023-10-15 17:12:50,254][52833] Updated weights for policy 0, policy_version 60630 (0.0011) -[2023-10-15 17:12:50,625][52833] Updated weights for policy 0, policy_version 60640 (0.0011) -[2023-10-15 17:12:51,134][52866] Updated weights for policy 1, policy_version 60840 (0.0008) -[2023-10-15 17:12:51,492][52866] Updated weights for policy 1, policy_version 60850 (0.0008) -[2023-10-15 17:12:51,866][52866] Updated weights for policy 1, policy_version 60860 (0.0007) -[2023-10-15 17:12:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 124420096. Throughput: 0: 1784.2, 1: 1802.0. Samples: 31111250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 17:12:53,441][51532] Avg episode reward: [(0, '54.150'), (1, '61.560')] -[2023-10-15 17:12:54,536][52833] Updated weights for policy 0, policy_version 60650 (0.0009) -[2023-10-15 17:12:54,912][52833] Updated weights for policy 0, policy_version 60660 (0.0009) -[2023-10-15 17:12:55,282][52833] Updated weights for policy 0, policy_version 60670 (0.0010) -[2023-10-15 17:12:55,566][52866] Updated weights for policy 1, policy_version 60870 (0.0008) -[2023-10-15 17:12:55,933][52866] Updated weights for policy 1, policy_version 60880 (0.0007) -[2023-10-15 17:12:56,302][52866] Updated weights for policy 1, policy_version 60890 (0.0008) -[2023-10-15 17:12:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 124485632. Throughput: 0: 1780.5, 1: 1804.6. Samples: 31133872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 17:12:58,442][51532] Avg episode reward: [(0, '51.110'), (1, '63.120')] -[2023-10-15 17:12:59,043][52833] Updated weights for policy 0, policy_version 60680 (0.0009) -[2023-10-15 17:12:59,428][52833] Updated weights for policy 0, policy_version 60690 (0.0008) -[2023-10-15 17:12:59,798][52833] Updated weights for policy 0, policy_version 60700 (0.0007) -[2023-10-15 17:13:00,072][52866] Updated weights for policy 1, policy_version 60900 (0.0011) -[2023-10-15 17:13:00,440][52866] Updated weights for policy 1, policy_version 60910 (0.0007) -[2023-10-15 17:13:00,801][52866] Updated weights for policy 1, policy_version 60920 (0.0010) -[2023-10-15 17:13:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 124551168. Throughput: 0: 1778.2, 1: 1812.1. Samples: 31143906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 17:13:03,441][51532] Avg episode reward: [(0, '52.490'), (1, '66.800')] -[2023-10-15 17:13:03,516][52833] Updated weights for policy 0, policy_version 60710 (0.0007) -[2023-10-15 17:13:03,874][52833] Updated weights for policy 0, policy_version 60720 (0.0008) -[2023-10-15 17:13:04,254][52833] Updated weights for policy 0, policy_version 60730 (0.0010) -[2023-10-15 17:13:04,487][52866] Updated weights for policy 1, policy_version 60930 (0.0009) -[2023-10-15 17:13:04,853][52866] Updated weights for policy 1, policy_version 60940 (0.0010) -[2023-10-15 17:13:05,222][52866] Updated weights for policy 1, policy_version 60950 (0.0008) -[2023-10-15 17:13:05,579][52866] Updated weights for policy 1, policy_version 60960 (0.0009) -[2023-10-15 17:13:08,027][52833] Updated weights for policy 0, policy_version 60740 (0.0009) -[2023-10-15 17:13:08,403][52833] Updated weights for policy 0, policy_version 60750 (0.0009) -[2023-10-15 17:13:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 124616704. Throughput: 0: 1773.5, 1: 1812.1. Samples: 31166080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 17:13:08,441][51532] Avg episode reward: [(0, '55.140'), (1, '69.240')] -[2023-10-15 17:13:08,442][52518] Saving new best policy, reward=69.240! -[2023-10-15 17:13:08,777][52833] Updated weights for policy 0, policy_version 60760 (0.0009) -[2023-10-15 17:13:09,224][52866] Updated weights for policy 1, policy_version 60970 (0.0008) -[2023-10-15 17:13:09,593][52866] Updated weights for policy 1, policy_version 60980 (0.0009) -[2023-10-15 17:13:09,972][52866] Updated weights for policy 1, policy_version 60990 (0.0008) -[2023-10-15 17:13:12,414][52833] Updated weights for policy 0, policy_version 60770 (0.0009) -[2023-10-15 17:13:12,793][52833] Updated weights for policy 0, policy_version 60780 (0.0007) -[2023-10-15 17:13:13,155][52833] Updated weights for policy 0, policy_version 60790 (0.0007) -[2023-10-15 17:13:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 124682240. Throughput: 0: 1797.8, 1: 1807.2. Samples: 31188148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 17:13:13,442][51532] Avg episode reward: [(0, '56.830'), (1, '65.900')] -[2023-10-15 17:13:13,527][52833] Updated weights for policy 0, policy_version 60800 (0.0010) -[2023-10-15 17:13:13,631][52866] Updated weights for policy 1, policy_version 61000 (0.0008) -[2023-10-15 17:13:13,992][52866] Updated weights for policy 1, policy_version 61010 (0.0007) -[2023-10-15 17:13:14,363][52866] Updated weights for policy 1, policy_version 61020 (0.0007) -[2023-10-15 17:13:17,393][52833] Updated weights for policy 0, policy_version 60810 (0.0008) -[2023-10-15 17:13:17,768][52833] Updated weights for policy 0, policy_version 60820 (0.0008) -[2023-10-15 17:13:18,134][52833] Updated weights for policy 0, policy_version 60830 (0.0007) -[2023-10-15 17:13:18,226][52866] Updated weights for policy 1, policy_version 61030 (0.0007) -[2023-10-15 17:13:18,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 124780544. Throughput: 0: 1779.0, 1: 1811.6. Samples: 31198602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-15 17:13:18,442][51532] Avg episode reward: [(0, '55.870'), (1, '66.410')] -[2023-10-15 17:13:18,602][52866] Updated weights for policy 1, policy_version 61040 (0.0010) -[2023-10-15 17:13:18,967][52866] Updated weights for policy 1, policy_version 61050 (0.0011) -[2023-10-15 17:13:21,826][52833] Updated weights for policy 0, policy_version 60840 (0.0009) -[2023-10-15 17:13:22,214][52833] Updated weights for policy 0, policy_version 60850 (0.0011) -[2023-10-15 17:13:22,579][52833] Updated weights for policy 0, policy_version 60860 (0.0009) -[2023-10-15 17:13:22,705][52866] Updated weights for policy 1, policy_version 61060 (0.0010) -[2023-10-15 17:13:23,079][52866] Updated weights for policy 1, policy_version 61070 (0.0011) -[2023-10-15 17:13:23,439][52866] Updated weights for policy 1, policy_version 61080 (0.0010) -[2023-10-15 17:13:23,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 124846080. Throughput: 0: 1797.0, 1: 1806.8. Samples: 31220514. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:13:23,442][51532] Avg episode reward: [(0, '56.010'), (1, '66.180')] -[2023-10-15 17:13:26,435][52833] Updated weights for policy 0, policy_version 60870 (0.0008) -[2023-10-15 17:13:26,810][52833] Updated weights for policy 0, policy_version 60880 (0.0008) -[2023-10-15 17:13:27,172][52833] Updated weights for policy 0, policy_version 60890 (0.0008) -[2023-10-15 17:13:27,237][52866] Updated weights for policy 1, policy_version 61090 (0.0008) -[2023-10-15 17:13:27,591][52866] Updated weights for policy 1, policy_version 61100 (0.0007) -[2023-10-15 17:13:27,949][52866] Updated weights for policy 1, policy_version 61110 (0.0007) -[2023-10-15 17:13:28,323][52866] Updated weights for policy 1, policy_version 61120 (0.0009) -[2023-10-15 17:13:28,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124944384. Throughput: 0: 1774.6, 1: 1817.2. Samples: 31240854. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:13:28,441][51532] Avg episode reward: [(0, '57.190'), (1, '66.830')] -[2023-10-15 17:13:31,088][52833] Updated weights for policy 0, policy_version 60900 (0.0010) -[2023-10-15 17:13:31,462][52833] Updated weights for policy 0, policy_version 60910 (0.0007) -[2023-10-15 17:13:31,826][52833] Updated weights for policy 0, policy_version 60920 (0.0007) -[2023-10-15 17:13:32,229][52866] Updated weights for policy 1, policy_version 61130 (0.0008) -[2023-10-15 17:13:32,605][52866] Updated weights for policy 1, policy_version 61140 (0.0008) -[2023-10-15 17:13:32,973][52866] Updated weights for policy 1, policy_version 61150 (0.0009) -[2023-10-15 17:13:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 125009920. Throughput: 0: 1800.7, 1: 1802.3. Samples: 31252980. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:13:33,442][51532] Avg episode reward: [(0, '59.430'), (1, '61.590')] -[2023-10-15 17:13:35,547][52833] Updated weights for policy 0, policy_version 60930 (0.0008) -[2023-10-15 17:13:35,922][52833] Updated weights for policy 0, policy_version 60940 (0.0008) -[2023-10-15 17:13:36,287][52833] Updated weights for policy 0, policy_version 60950 (0.0009) -[2023-10-15 17:13:36,652][52833] Updated weights for policy 0, policy_version 60960 (0.0008) -[2023-10-15 17:13:36,710][52866] Updated weights for policy 1, policy_version 61160 (0.0007) -[2023-10-15 17:13:37,076][52866] Updated weights for policy 1, policy_version 61170 (0.0007) -[2023-10-15 17:13:37,438][52866] Updated weights for policy 1, policy_version 61180 (0.0008) -[2023-10-15 17:13:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 125075456. Throughput: 0: 1778.4, 1: 1819.7. Samples: 31273164. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:13:38,442][51532] Avg episode reward: [(0, '53.980'), (1, '60.470')] -[2023-10-15 17:13:40,121][52833] Updated weights for policy 0, policy_version 60970 (0.0007) -[2023-10-15 17:13:40,479][52833] Updated weights for policy 0, policy_version 60980 (0.0008) -[2023-10-15 17:13:40,865][52833] Updated weights for policy 0, policy_version 60990 (0.0009) -[2023-10-15 17:13:41,241][52866] Updated weights for policy 1, policy_version 61190 (0.0009) -[2023-10-15 17:13:41,606][52866] Updated weights for policy 1, policy_version 61200 (0.0008) -[2023-10-15 17:13:41,968][52866] Updated weights for policy 1, policy_version 61210 (0.0008) -[2023-10-15 17:13:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125140992. Throughput: 0: 1787.6, 1: 1797.7. Samples: 31295206. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:13:43,441][51532] Avg episode reward: [(0, '53.400'), (1, '58.160')] -[2023-10-15 17:13:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000060992_62455808.pth... -[2023-10-15 17:13:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000061216_62685184.pth... -[2023-10-15 17:13:43,480][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000059328_60751872.pth -[2023-10-15 17:13:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000059520_60948480.pth -[2023-10-15 17:13:44,659][52833] Updated weights for policy 0, policy_version 61000 (0.0009) -[2023-10-15 17:13:45,028][52833] Updated weights for policy 0, policy_version 61010 (0.0009) -[2023-10-15 17:13:45,408][52833] Updated weights for policy 0, policy_version 61020 (0.0009) -[2023-10-15 17:13:45,622][52866] Updated weights for policy 1, policy_version 61220 (0.0007) -[2023-10-15 17:13:45,989][52866] Updated weights for policy 1, policy_version 61230 (0.0008) -[2023-10-15 17:13:46,349][52866] Updated weights for policy 1, policy_version 61240 (0.0010) -[2023-10-15 17:13:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 125206528. Throughput: 0: 1787.0, 1: 1811.7. Samples: 31305848. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:13:48,442][51532] Avg episode reward: [(0, '57.600'), (1, '57.570')] -[2023-10-15 17:13:49,228][52833] Updated weights for policy 0, policy_version 61030 (0.0009) -[2023-10-15 17:13:49,611][52833] Updated weights for policy 0, policy_version 61040 (0.0008) -[2023-10-15 17:13:49,988][52833] Updated weights for policy 0, policy_version 61050 (0.0008) -[2023-10-15 17:13:50,094][52866] Updated weights for policy 1, policy_version 61250 (0.0008) -[2023-10-15 17:13:50,452][52866] Updated weights for policy 1, policy_version 61260 (0.0008) -[2023-10-15 17:13:50,818][52866] Updated weights for policy 1, policy_version 61270 (0.0011) -[2023-10-15 17:13:51,185][52866] Updated weights for policy 1, policy_version 61280 (0.0009) -[2023-10-15 17:13:53,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 125272064. Throughput: 0: 1787.0, 1: 1796.1. Samples: 31327322. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:13:53,443][51532] Avg episode reward: [(0, '53.120'), (1, '56.420')] -[2023-10-15 17:13:53,674][52833] Updated weights for policy 0, policy_version 61060 (0.0007) -[2023-10-15 17:13:54,039][52833] Updated weights for policy 0, policy_version 61070 (0.0008) -[2023-10-15 17:13:54,415][52833] Updated weights for policy 0, policy_version 61080 (0.0010) -[2023-10-15 17:13:54,903][52866] Updated weights for policy 1, policy_version 61290 (0.0008) -[2023-10-15 17:13:55,271][52866] Updated weights for policy 1, policy_version 61300 (0.0008) -[2023-10-15 17:13:55,637][52866] Updated weights for policy 1, policy_version 61310 (0.0009) -[2023-10-15 17:13:58,206][52833] Updated weights for policy 0, policy_version 61090 (0.0008) -[2023-10-15 17:13:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 125337600. Throughput: 0: 1801.5, 1: 1797.7. Samples: 31350112. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:13:58,441][51532] Avg episode reward: [(0, '51.670'), (1, '53.510')] -[2023-10-15 17:13:58,570][52833] Updated weights for policy 0, policy_version 61100 (0.0007) -[2023-10-15 17:13:58,937][52833] Updated weights for policy 0, policy_version 61110 (0.0008) -[2023-10-15 17:13:59,309][52833] Updated weights for policy 0, policy_version 61120 (0.0008) -[2023-10-15 17:13:59,323][52866] Updated weights for policy 1, policy_version 61320 (0.0008) -[2023-10-15 17:13:59,682][52866] Updated weights for policy 1, policy_version 61330 (0.0008) -[2023-10-15 17:14:00,052][52866] Updated weights for policy 1, policy_version 61340 (0.0007) -[2023-10-15 17:14:03,192][52833] Updated weights for policy 0, policy_version 61130 (0.0007) -[2023-10-15 17:14:03,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 125403136. Throughput: 0: 1786.4, 1: 1796.6. Samples: 31359836. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-15 17:14:03,441][51532] Avg episode reward: [(0, '52.570'), (1, '53.930')] -[2023-10-15 17:14:03,556][52833] Updated weights for policy 0, policy_version 61140 (0.0008) -[2023-10-15 17:14:03,649][52866] Updated weights for policy 1, policy_version 61350 (0.0007) -[2023-10-15 17:14:03,925][52833] Updated weights for policy 0, policy_version 61150 (0.0007) -[2023-10-15 17:14:04,006][52866] Updated weights for policy 1, policy_version 61360 (0.0008) -[2023-10-15 17:14:04,376][52866] Updated weights for policy 1, policy_version 61370 (0.0007) -[2023-10-15 17:14:07,591][52833] Updated weights for policy 0, policy_version 61160 (0.0010) -[2023-10-15 17:14:07,955][52833] Updated weights for policy 0, policy_version 61170 (0.0008) -[2023-10-15 17:14:08,039][52866] Updated weights for policy 1, policy_version 61380 (0.0008) -[2023-10-15 17:14:08,331][52833] Updated weights for policy 0, policy_version 61180 (0.0008) -[2023-10-15 17:14:08,406][52866] Updated weights for policy 1, policy_version 61390 (0.0008) -[2023-10-15 17:14:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 125468672. Throughput: 0: 1799.5, 1: 1802.6. Samples: 31382606. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) -[2023-10-15 17:14:08,441][51532] Avg episode reward: [(0, '52.420'), (1, '54.140')] -[2023-10-15 17:14:08,770][52866] Updated weights for policy 1, policy_version 61400 (0.0008) -[2023-10-15 17:14:12,055][52833] Updated weights for policy 0, policy_version 61190 (0.0008) -[2023-10-15 17:14:12,429][52833] Updated weights for policy 0, policy_version 61200 (0.0009) -[2023-10-15 17:14:12,514][52866] Updated weights for policy 1, policy_version 61410 (0.0009) -[2023-10-15 17:14:12,789][52833] Updated weights for policy 0, policy_version 61210 (0.0007) -[2023-10-15 17:14:12,882][52866] Updated weights for policy 1, policy_version 61420 (0.0007) -[2023-10-15 17:14:13,251][52866] Updated weights for policy 1, policy_version 61430 (0.0008) -[2023-10-15 17:14:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 125566976. Throughput: 0: 1797.3, 1: 1811.2. Samples: 31403236. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) -[2023-10-15 17:14:13,441][51532] Avg episode reward: [(0, '47.940'), (1, '53.530')] -[2023-10-15 17:14:13,615][52866] Updated weights for policy 1, policy_version 61440 (0.0007) -[2023-10-15 17:14:16,397][52833] Updated weights for policy 0, policy_version 61220 (0.0007) -[2023-10-15 17:14:16,770][52833] Updated weights for policy 0, policy_version 61230 (0.0008) -[2023-10-15 17:14:17,135][52833] Updated weights for policy 0, policy_version 61240 (0.0008) -[2023-10-15 17:14:17,332][52866] Updated weights for policy 1, policy_version 61450 (0.0007) -[2023-10-15 17:14:17,697][52866] Updated weights for policy 1, policy_version 61460 (0.0008) -[2023-10-15 17:14:18,064][52866] Updated weights for policy 1, policy_version 61470 (0.0008) -[2023-10-15 17:14:18,441][51532] Fps is (10 sec: 19660.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125665280. Throughput: 0: 1798.1, 1: 1809.0. Samples: 31415298. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) -[2023-10-15 17:14:18,441][51532] Avg episode reward: [(0, '48.210'), (1, '53.980')] -[2023-10-15 17:14:21,051][52833] Updated weights for policy 0, policy_version 61250 (0.0009) -[2023-10-15 17:14:21,416][52833] Updated weights for policy 0, policy_version 61260 (0.0011) -[2023-10-15 17:14:21,783][52833] Updated weights for policy 0, policy_version 61270 (0.0008) -[2023-10-15 17:14:21,818][52866] Updated weights for policy 1, policy_version 61480 (0.0007) -[2023-10-15 17:14:22,147][52833] Updated weights for policy 0, policy_version 61280 (0.0009) -[2023-10-15 17:14:22,185][52866] Updated weights for policy 1, policy_version 61490 (0.0007) -[2023-10-15 17:14:22,552][52866] Updated weights for policy 1, policy_version 61500 (0.0008) -[2023-10-15 17:14:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 125730816. Throughput: 0: 1805.4, 1: 1810.9. Samples: 31435900. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) -[2023-10-15 17:14:23,441][51532] Avg episode reward: [(0, '48.520'), (1, '55.480')] -[2023-10-15 17:14:25,880][52833] Updated weights for policy 0, policy_version 61290 (0.0011) -[2023-10-15 17:14:26,166][52866] Updated weights for policy 1, policy_version 61510 (0.0007) -[2023-10-15 17:14:26,257][52833] Updated weights for policy 0, policy_version 61300 (0.0010) -[2023-10-15 17:14:26,536][52866] Updated weights for policy 1, policy_version 61520 (0.0008) -[2023-10-15 17:14:26,618][52833] Updated weights for policy 0, policy_version 61310 (0.0008) -[2023-10-15 17:14:26,887][52866] Updated weights for policy 1, policy_version 61530 (0.0010) -[2023-10-15 17:14:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 125796352. Throughput: 0: 1787.4, 1: 1814.4. Samples: 31457290. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) -[2023-10-15 17:14:28,442][51532] Avg episode reward: [(0, '50.630'), (1, '56.790')] -[2023-10-15 17:14:30,275][52833] Updated weights for policy 0, policy_version 61320 (0.0008) -[2023-10-15 17:14:30,620][52866] Updated weights for policy 1, policy_version 61540 (0.0009) -[2023-10-15 17:14:30,642][52833] Updated weights for policy 0, policy_version 61330 (0.0007) -[2023-10-15 17:14:30,987][52866] Updated weights for policy 1, policy_version 61550 (0.0007) -[2023-10-15 17:14:30,998][52833] Updated weights for policy 0, policy_version 61340 (0.0007) -[2023-10-15 17:14:31,349][52866] Updated weights for policy 1, policy_version 61560 (0.0009) -[2023-10-15 17:14:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125861888. Throughput: 0: 1800.0, 1: 1810.7. Samples: 31468330. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) -[2023-10-15 17:14:33,442][51532] Avg episode reward: [(0, '50.390'), (1, '59.840')] -[2023-10-15 17:14:34,699][52833] Updated weights for policy 0, policy_version 61350 (0.0008) -[2023-10-15 17:14:35,029][52866] Updated weights for policy 1, policy_version 61570 (0.0007) -[2023-10-15 17:14:35,073][52833] Updated weights for policy 0, policy_version 61360 (0.0007) -[2023-10-15 17:14:35,399][52866] Updated weights for policy 1, policy_version 61580 (0.0007) -[2023-10-15 17:14:35,437][52833] Updated weights for policy 0, policy_version 61370 (0.0007) -[2023-10-15 17:14:35,769][52866] Updated weights for policy 1, policy_version 61590 (0.0009) -[2023-10-15 17:14:36,138][52866] Updated weights for policy 1, policy_version 61600 (0.0007) -[2023-10-15 17:14:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 125927424. Throughput: 0: 1797.4, 1: 1812.0. Samples: 31489742. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) -[2023-10-15 17:14:38,442][51532] Avg episode reward: [(0, '51.530'), (1, '59.610')] -[2023-10-15 17:14:39,087][52833] Updated weights for policy 0, policy_version 61380 (0.0010) -[2023-10-15 17:14:39,463][52833] Updated weights for policy 0, policy_version 61390 (0.0009) -[2023-10-15 17:14:39,828][52833] Updated weights for policy 0, policy_version 61400 (0.0007) -[2023-10-15 17:14:39,847][52866] Updated weights for policy 1, policy_version 61610 (0.0009) -[2023-10-15 17:14:40,205][52866] Updated weights for policy 1, policy_version 61620 (0.0008) -[2023-10-15 17:14:40,569][52866] Updated weights for policy 1, policy_version 61630 (0.0007) -[2023-10-15 17:14:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 125992960. Throughput: 0: 1796.6, 1: 1806.8. Samples: 31512268. Policy #0 lag: (min: 29.0, avg: 31.3, max: 61.0) -[2023-10-15 17:14:43,441][51532] Avg episode reward: [(0, '54.580'), (1, '59.250')] -[2023-10-15 17:14:43,591][52833] Updated weights for policy 0, policy_version 61410 (0.0007) -[2023-10-15 17:14:43,961][52833] Updated weights for policy 0, policy_version 61420 (0.0008) -[2023-10-15 17:14:44,333][52833] Updated weights for policy 0, policy_version 61430 (0.0007) -[2023-10-15 17:14:44,493][52866] Updated weights for policy 1, policy_version 61640 (0.0009) -[2023-10-15 17:14:44,707][52833] Updated weights for policy 0, policy_version 61440 (0.0009) -[2023-10-15 17:14:44,865][52866] Updated weights for policy 1, policy_version 61650 (0.0008) -[2023-10-15 17:14:45,225][52866] Updated weights for policy 1, policy_version 61660 (0.0008) -[2023-10-15 17:14:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 126058496. Throughput: 0: 1797.8, 1: 1807.9. Samples: 31522092. Policy #0 lag: (min: 12.0, avg: 14.8, max: 44.0) -[2023-10-15 17:14:48,441][51532] Avg episode reward: [(0, '52.390'), (1, '57.890')] -[2023-10-15 17:14:48,593][52833] Updated weights for policy 0, policy_version 61450 (0.0010) -[2023-10-15 17:14:48,965][52833] Updated weights for policy 0, policy_version 61460 (0.0010) -[2023-10-15 17:14:49,019][52866] Updated weights for policy 1, policy_version 61670 (0.0008) -[2023-10-15 17:14:49,327][52833] Updated weights for policy 0, policy_version 61470 (0.0007) -[2023-10-15 17:14:49,392][52866] Updated weights for policy 1, policy_version 61680 (0.0007) -[2023-10-15 17:14:49,762][52866] Updated weights for policy 1, policy_version 61690 (0.0010) -[2023-10-15 17:14:52,864][52833] Updated weights for policy 0, policy_version 61480 (0.0009) -[2023-10-15 17:14:53,232][52833] Updated weights for policy 0, policy_version 61490 (0.0008) -[2023-10-15 17:14:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14329.1). Total num frames: 126124032. Throughput: 0: 1794.7, 1: 1801.6. Samples: 31544442. Policy #0 lag: (min: 12.0, avg: 14.8, max: 44.0) -[2023-10-15 17:14:53,441][51532] Avg episode reward: [(0, '51.370'), (1, '61.360')] -[2023-10-15 17:14:53,513][52866] Updated weights for policy 1, policy_version 61700 (0.0008) -[2023-10-15 17:14:53,601][52833] Updated weights for policy 0, policy_version 61500 (0.0007) -[2023-10-15 17:14:53,874][52866] Updated weights for policy 1, policy_version 61710 (0.0008) -[2023-10-15 17:14:54,248][52866] Updated weights for policy 1, policy_version 61720 (0.0008) -[2023-10-15 17:14:57,418][52833] Updated weights for policy 0, policy_version 61510 (0.0008) -[2023-10-15 17:14:57,790][52833] Updated weights for policy 0, policy_version 61520 (0.0007) -[2023-10-15 17:14:58,097][52866] Updated weights for policy 1, policy_version 61730 (0.0008) -[2023-10-15 17:14:58,161][52833] Updated weights for policy 0, policy_version 61530 (0.0009) -[2023-10-15 17:14:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 126222336. Throughput: 0: 1809.2, 1: 1810.7. Samples: 31566134. Policy #0 lag: (min: 12.0, avg: 14.8, max: 44.0) -[2023-10-15 17:14:58,442][51532] Avg episode reward: [(0, '51.690'), (1, '60.790')] -[2023-10-15 17:14:58,465][52866] Updated weights for policy 1, policy_version 61740 (0.0008) -[2023-10-15 17:14:58,828][52866] Updated weights for policy 1, policy_version 61750 (0.0007) -[2023-10-15 17:14:59,201][52866] Updated weights for policy 1, policy_version 61760 (0.0009) -[2023-10-15 17:15:01,962][52833] Updated weights for policy 0, policy_version 61540 (0.0007) -[2023-10-15 17:15:02,342][52833] Updated weights for policy 0, policy_version 61550 (0.0007) -[2023-10-15 17:15:02,717][52833] Updated weights for policy 0, policy_version 61560 (0.0010) -[2023-10-15 17:15:03,119][52866] Updated weights for policy 1, policy_version 61770 (0.0007) -[2023-10-15 17:15:03,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 126287872. Throughput: 0: 1794.5, 1: 1792.3. Samples: 31576704. Policy #0 lag: (min: 12.0, avg: 14.8, max: 44.0) -[2023-10-15 17:15:03,442][51532] Avg episode reward: [(0, '52.800'), (1, '59.540')] -[2023-10-15 17:15:03,496][52866] Updated weights for policy 1, policy_version 61780 (0.0009) -[2023-10-15 17:15:03,862][52866] Updated weights for policy 1, policy_version 61790 (0.0008) -[2023-10-15 17:15:06,486][52833] Updated weights for policy 0, policy_version 61570 (0.0008) -[2023-10-15 17:15:06,849][52833] Updated weights for policy 0, policy_version 61580 (0.0010) -[2023-10-15 17:15:07,221][52833] Updated weights for policy 0, policy_version 61590 (0.0007) -[2023-10-15 17:15:07,445][52866] Updated weights for policy 1, policy_version 61800 (0.0008) -[2023-10-15 17:15:07,588][52833] Updated weights for policy 0, policy_version 61600 (0.0008) -[2023-10-15 17:15:07,809][52866] Updated weights for policy 1, policy_version 61810 (0.0007) -[2023-10-15 17:15:08,174][52866] Updated weights for policy 1, policy_version 61820 (0.0010) -[2023-10-15 17:15:08,441][51532] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 126386176. Throughput: 0: 1809.9, 1: 1805.6. Samples: 31598596. Policy #0 lag: (min: 12.0, avg: 14.8, max: 44.0) -[2023-10-15 17:15:08,442][51532] Avg episode reward: [(0, '54.290'), (1, '60.890')] -[2023-10-15 17:15:11,363][52833] Updated weights for policy 0, policy_version 61610 (0.0011) -[2023-10-15 17:15:11,727][52833] Updated weights for policy 0, policy_version 61620 (0.0010) -[2023-10-15 17:15:11,959][52866] Updated weights for policy 1, policy_version 61830 (0.0009) -[2023-10-15 17:15:12,086][52833] Updated weights for policy 0, policy_version 61630 (0.0008) -[2023-10-15 17:15:12,329][52866] Updated weights for policy 1, policy_version 61840 (0.0009) -[2023-10-15 17:15:12,693][52866] Updated weights for policy 1, policy_version 61850 (0.0009) -[2023-10-15 17:15:13,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 126451712. Throughput: 0: 1801.3, 1: 1784.8. Samples: 31618664. Policy #0 lag: (min: 12.0, avg: 14.8, max: 44.0) -[2023-10-15 17:15:13,441][51532] Avg episode reward: [(0, '55.030'), (1, '61.030')] -[2023-10-15 17:15:15,785][52833] Updated weights for policy 0, policy_version 61640 (0.0009) -[2023-10-15 17:15:16,156][52833] Updated weights for policy 0, policy_version 61650 (0.0009) -[2023-10-15 17:15:16,509][52866] Updated weights for policy 1, policy_version 61860 (0.0009) -[2023-10-15 17:15:16,525][52833] Updated weights for policy 0, policy_version 61660 (0.0010) -[2023-10-15 17:15:16,880][52866] Updated weights for policy 1, policy_version 61870 (0.0008) -[2023-10-15 17:15:17,249][52866] Updated weights for policy 1, policy_version 61880 (0.0008) -[2023-10-15 17:15:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126517248. Throughput: 0: 1814.8, 1: 1801.0. Samples: 31631042. Policy #0 lag: (min: 12.0, avg: 14.8, max: 44.0) -[2023-10-15 17:15:18,441][51532] Avg episode reward: [(0, '56.120'), (1, '59.540')] -[2023-10-15 17:15:20,243][52833] Updated weights for policy 0, policy_version 61670 (0.0008) -[2023-10-15 17:15:20,605][52833] Updated weights for policy 0, policy_version 61680 (0.0009) -[2023-10-15 17:15:20,955][52866] Updated weights for policy 1, policy_version 61890 (0.0008) -[2023-10-15 17:15:20,973][52833] Updated weights for policy 0, policy_version 61690 (0.0008) -[2023-10-15 17:15:21,321][52866] Updated weights for policy 1, policy_version 61900 (0.0009) -[2023-10-15 17:15:21,695][52866] Updated weights for policy 1, policy_version 61910 (0.0009) -[2023-10-15 17:15:22,053][52866] Updated weights for policy 1, policy_version 61920 (0.0008) -[2023-10-15 17:15:23,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 126582784. Throughput: 0: 1798.3, 1: 1793.1. Samples: 31651356. Policy #0 lag: (min: 12.0, avg: 14.8, max: 44.0) -[2023-10-15 17:15:23,442][51532] Avg episode reward: [(0, '52.020'), (1, '57.000')] -[2023-10-15 17:15:24,435][52833] Updated weights for policy 0, policy_version 61700 (0.0008) -[2023-10-15 17:15:24,801][52833] Updated weights for policy 0, policy_version 61710 (0.0008) -[2023-10-15 17:15:25,177][52833] Updated weights for policy 0, policy_version 61720 (0.0008) -[2023-10-15 17:15:25,802][52866] Updated weights for policy 1, policy_version 61930 (0.0007) -[2023-10-15 17:15:26,175][52866] Updated weights for policy 1, policy_version 61940 (0.0007) -[2023-10-15 17:15:26,539][52866] Updated weights for policy 1, policy_version 61950 (0.0010) -[2023-10-15 17:15:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 126648320. Throughput: 0: 1801.7, 1: 1791.4. Samples: 31673956. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-15 17:15:28,442][51532] Avg episode reward: [(0, '48.390'), (1, '60.060')] -[2023-10-15 17:15:28,959][52833] Updated weights for policy 0, policy_version 61730 (0.0009) -[2023-10-15 17:15:29,331][52833] Updated weights for policy 0, policy_version 61740 (0.0008) -[2023-10-15 17:15:29,706][52833] Updated weights for policy 0, policy_version 61750 (0.0011) -[2023-10-15 17:15:30,077][52833] Updated weights for policy 0, policy_version 61760 (0.0009) -[2023-10-15 17:15:30,139][52866] Updated weights for policy 1, policy_version 61960 (0.0008) -[2023-10-15 17:15:30,500][52866] Updated weights for policy 1, policy_version 61970 (0.0007) -[2023-10-15 17:15:30,867][52866] Updated weights for policy 1, policy_version 61980 (0.0008) -[2023-10-15 17:15:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 126713856. Throughput: 0: 1802.1, 1: 1793.3. Samples: 31683886. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-15 17:15:33,443][51532] Avg episode reward: [(0, '50.950'), (1, '57.950')] -[2023-10-15 17:15:33,884][52833] Updated weights for policy 0, policy_version 61770 (0.0009) -[2023-10-15 17:15:34,261][52833] Updated weights for policy 0, policy_version 61780 (0.0010) -[2023-10-15 17:15:34,606][52866] Updated weights for policy 1, policy_version 61990 (0.0007) -[2023-10-15 17:15:34,624][52833] Updated weights for policy 0, policy_version 61790 (0.0010) -[2023-10-15 17:15:34,958][52866] Updated weights for policy 1, policy_version 62000 (0.0007) -[2023-10-15 17:15:35,320][52866] Updated weights for policy 1, policy_version 62010 (0.0007) -[2023-10-15 17:15:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 126779392. Throughput: 0: 1800.5, 1: 1791.1. Samples: 31706064. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-15 17:15:38,441][51532] Avg episode reward: [(0, '50.110'), (1, '57.680')] -[2023-10-15 17:15:38,478][52833] Updated weights for policy 0, policy_version 61800 (0.0008) -[2023-10-15 17:15:38,851][52833] Updated weights for policy 0, policy_version 61810 (0.0009) -[2023-10-15 17:15:39,102][52866] Updated weights for policy 1, policy_version 62020 (0.0007) -[2023-10-15 17:15:39,222][52833] Updated weights for policy 0, policy_version 61820 (0.0007) -[2023-10-15 17:15:39,469][52866] Updated weights for policy 1, policy_version 62030 (0.0008) -[2023-10-15 17:15:39,843][52866] Updated weights for policy 1, policy_version 62040 (0.0008) -[2023-10-15 17:15:42,938][52833] Updated weights for policy 0, policy_version 61830 (0.0007) -[2023-10-15 17:15:43,305][52833] Updated weights for policy 0, policy_version 61840 (0.0007) -[2023-10-15 17:15:43,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 126844928. Throughput: 0: 1811.5, 1: 1794.8. Samples: 31728416. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-15 17:15:43,441][51532] Avg episode reward: [(0, '50.530'), (1, '58.270')] -[2023-10-15 17:15:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000062048_63537152.pth... -[2023-10-15 17:15:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000060384_61833216.pth -[2023-10-15 17:15:43,674][52833] Updated weights for policy 0, policy_version 61850 (0.0007) -[2023-10-15 17:15:43,704][52866] Updated weights for policy 1, policy_version 62050 (0.0007) -[2023-10-15 17:15:43,895][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000061856_63340544.pth... -[2023-10-15 17:15:43,924][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000060160_61603840.pth -[2023-10-15 17:15:44,067][52866] Updated weights for policy 1, policy_version 62060 (0.0012) -[2023-10-15 17:15:44,430][52866] Updated weights for policy 1, policy_version 62070 (0.0008) -[2023-10-15 17:15:44,801][52866] Updated weights for policy 1, policy_version 62080 (0.0008) -[2023-10-15 17:15:47,510][52833] Updated weights for policy 0, policy_version 61860 (0.0009) -[2023-10-15 17:15:47,878][52833] Updated weights for policy 0, policy_version 61870 (0.0010) -[2023-10-15 17:15:48,239][52833] Updated weights for policy 0, policy_version 61880 (0.0008) -[2023-10-15 17:15:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 126910464. Throughput: 0: 1800.2, 1: 1797.4. Samples: 31738598. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-15 17:15:48,441][51532] Avg episode reward: [(0, '51.180'), (1, '57.000')] -[2023-10-15 17:15:48,618][52866] Updated weights for policy 1, policy_version 62090 (0.0009) -[2023-10-15 17:15:48,983][52866] Updated weights for policy 1, policy_version 62100 (0.0011) -[2023-10-15 17:15:49,349][52866] Updated weights for policy 1, policy_version 62110 (0.0007) -[2023-10-15 17:15:52,068][52833] Updated weights for policy 0, policy_version 61890 (0.0009) -[2023-10-15 17:15:52,440][52833] Updated weights for policy 0, policy_version 61900 (0.0008) -[2023-10-15 17:15:52,808][52833] Updated weights for policy 0, policy_version 61910 (0.0008) -[2023-10-15 17:15:53,057][52866] Updated weights for policy 1, policy_version 62120 (0.0009) -[2023-10-15 17:15:53,170][52833] Updated weights for policy 0, policy_version 61920 (0.0007) -[2023-10-15 17:15:53,419][52866] Updated weights for policy 1, policy_version 62130 (0.0009) -[2023-10-15 17:15:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 127008768. Throughput: 0: 1808.9, 1: 1791.5. Samples: 31760616. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-15 17:15:53,441][51532] Avg episode reward: [(0, '49.190'), (1, '58.580')] -[2023-10-15 17:15:53,781][52866] Updated weights for policy 1, policy_version 62140 (0.0007) -[2023-10-15 17:15:56,950][52833] Updated weights for policy 0, policy_version 61930 (0.0008) -[2023-10-15 17:15:57,313][52833] Updated weights for policy 0, policy_version 61940 (0.0008) -[2023-10-15 17:15:57,549][52866] Updated weights for policy 1, policy_version 62150 (0.0007) -[2023-10-15 17:15:57,678][52833] Updated weights for policy 0, policy_version 61950 (0.0007) -[2023-10-15 17:15:57,922][52866] Updated weights for policy 1, policy_version 62160 (0.0007) -[2023-10-15 17:15:58,289][52866] Updated weights for policy 1, policy_version 62170 (0.0009) -[2023-10-15 17:15:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 127074304. Throughput: 0: 1789.4, 1: 1814.8. Samples: 31780852. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-15 17:15:58,442][51532] Avg episode reward: [(0, '49.210'), (1, '58.280')] -[2023-10-15 17:16:01,657][52833] Updated weights for policy 0, policy_version 61960 (0.0008) -[2023-10-15 17:16:02,021][52833] Updated weights for policy 0, policy_version 61970 (0.0008) -[2023-10-15 17:16:02,045][52866] Updated weights for policy 1, policy_version 62180 (0.0008) -[2023-10-15 17:16:02,391][52833] Updated weights for policy 0, policy_version 61980 (0.0007) -[2023-10-15 17:16:02,406][52866] Updated weights for policy 1, policy_version 62190 (0.0010) -[2023-10-15 17:16:02,771][52866] Updated weights for policy 1, policy_version 62200 (0.0008) -[2023-10-15 17:16:03,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 127172608. Throughput: 0: 1794.3, 1: 1797.6. Samples: 31792676. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-15 17:16:03,442][51532] Avg episode reward: [(0, '49.340'), (1, '58.490')] -[2023-10-15 17:16:06,022][52833] Updated weights for policy 0, policy_version 61990 (0.0009) -[2023-10-15 17:16:06,384][52833] Updated weights for policy 0, policy_version 62000 (0.0007) -[2023-10-15 17:16:06,469][52866] Updated weights for policy 1, policy_version 62210 (0.0008) -[2023-10-15 17:16:06,759][52833] Updated weights for policy 0, policy_version 62010 (0.0007) -[2023-10-15 17:16:06,834][52866] Updated weights for policy 1, policy_version 62220 (0.0008) -[2023-10-15 17:16:07,203][52866] Updated weights for policy 1, policy_version 62230 (0.0007) -[2023-10-15 17:16:07,565][52866] Updated weights for policy 1, policy_version 62240 (0.0007) -[2023-10-15 17:16:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 127238144. Throughput: 0: 1788.0, 1: 1809.5. Samples: 31813242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:16:08,442][51532] Avg episode reward: [(0, '48.420'), (1, '61.100')] -[2023-10-15 17:16:10,494][52833] Updated weights for policy 0, policy_version 62020 (0.0010) -[2023-10-15 17:16:10,869][52833] Updated weights for policy 0, policy_version 62030 (0.0008) -[2023-10-15 17:16:11,233][52833] Updated weights for policy 0, policy_version 62040 (0.0008) -[2023-10-15 17:16:11,411][52866] Updated weights for policy 1, policy_version 62250 (0.0008) -[2023-10-15 17:16:11,772][52866] Updated weights for policy 1, policy_version 62260 (0.0008) -[2023-10-15 17:16:12,145][52866] Updated weights for policy 1, policy_version 62270 (0.0008) -[2023-10-15 17:16:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 127303680. Throughput: 0: 1783.2, 1: 1794.4. Samples: 31834948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:16:13,442][51532] Avg episode reward: [(0, '52.150'), (1, '62.230')] -[2023-10-15 17:16:14,972][52833] Updated weights for policy 0, policy_version 62050 (0.0009) -[2023-10-15 17:16:15,347][52833] Updated weights for policy 0, policy_version 62060 (0.0008) -[2023-10-15 17:16:15,714][52833] Updated weights for policy 0, policy_version 62070 (0.0009) -[2023-10-15 17:16:15,733][52866] Updated weights for policy 1, policy_version 62280 (0.0007) -[2023-10-15 17:16:16,084][52833] Updated weights for policy 0, policy_version 62080 (0.0008) -[2023-10-15 17:16:16,101][52866] Updated weights for policy 1, policy_version 62290 (0.0007) -[2023-10-15 17:16:16,473][52866] Updated weights for policy 1, policy_version 62300 (0.0007) -[2023-10-15 17:16:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 127369216. Throughput: 0: 1788.2, 1: 1814.1. Samples: 31845986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:16:18,442][51532] Avg episode reward: [(0, '51.700'), (1, '60.840')] -[2023-10-15 17:16:19,929][52833] Updated weights for policy 0, policy_version 62090 (0.0008) -[2023-10-15 17:16:20,207][52866] Updated weights for policy 1, policy_version 62310 (0.0007) -[2023-10-15 17:16:20,290][52833] Updated weights for policy 0, policy_version 62100 (0.0008) -[2023-10-15 17:16:20,565][52866] Updated weights for policy 1, policy_version 62320 (0.0008) -[2023-10-15 17:16:20,657][52833] Updated weights for policy 0, policy_version 62110 (0.0007) -[2023-10-15 17:16:20,941][52866] Updated weights for policy 1, policy_version 62330 (0.0008) -[2023-10-15 17:16:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 127434752. Throughput: 0: 1779.6, 1: 1794.7. Samples: 31866908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:16:23,442][51532] Avg episode reward: [(0, '52.550'), (1, '64.790')] -[2023-10-15 17:16:24,404][52833] Updated weights for policy 0, policy_version 62120 (0.0008) -[2023-10-15 17:16:24,599][52866] Updated weights for policy 1, policy_version 62340 (0.0009) -[2023-10-15 17:16:24,775][52833] Updated weights for policy 0, policy_version 62130 (0.0010) -[2023-10-15 17:16:24,975][52866] Updated weights for policy 1, policy_version 62350 (0.0008) -[2023-10-15 17:16:25,132][52833] Updated weights for policy 0, policy_version 62140 (0.0008) -[2023-10-15 17:16:25,345][52866] Updated weights for policy 1, policy_version 62360 (0.0007) -[2023-10-15 17:16:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 127500288. Throughput: 0: 1782.7, 1: 1797.8. Samples: 31889538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:16:28,442][51532] Avg episode reward: [(0, '55.490'), (1, '64.370')] -[2023-10-15 17:16:28,844][52833] Updated weights for policy 0, policy_version 62150 (0.0007) -[2023-10-15 17:16:29,073][52866] Updated weights for policy 1, policy_version 62370 (0.0008) -[2023-10-15 17:16:29,213][52833] Updated weights for policy 0, policy_version 62160 (0.0007) -[2023-10-15 17:16:29,443][52866] Updated weights for policy 1, policy_version 62380 (0.0008) -[2023-10-15 17:16:29,589][52833] Updated weights for policy 0, policy_version 62170 (0.0008) -[2023-10-15 17:16:29,810][52866] Updated weights for policy 1, policy_version 62390 (0.0009) -[2023-10-15 17:16:30,179][52866] Updated weights for policy 1, policy_version 62400 (0.0008) -[2023-10-15 17:16:33,355][52833] Updated weights for policy 0, policy_version 62180 (0.0008) -[2023-10-15 17:16:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14329.1). Total num frames: 127565824. Throughput: 0: 1776.5, 1: 1796.1. Samples: 31899366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:16:33,441][51532] Avg episode reward: [(0, '55.030'), (1, '64.220')] -[2023-10-15 17:16:33,725][52833] Updated weights for policy 0, policy_version 62190 (0.0009) -[2023-10-15 17:16:34,026][52866] Updated weights for policy 1, policy_version 62410 (0.0008) -[2023-10-15 17:16:34,087][52833] Updated weights for policy 0, policy_version 62200 (0.0009) -[2023-10-15 17:16:34,383][52866] Updated weights for policy 1, policy_version 62420 (0.0007) -[2023-10-15 17:16:34,752][52866] Updated weights for policy 1, policy_version 62430 (0.0009) -[2023-10-15 17:16:37,788][52833] Updated weights for policy 0, policy_version 62210 (0.0008) -[2023-10-15 17:16:38,149][52833] Updated weights for policy 0, policy_version 62220 (0.0009) -[2023-10-15 17:16:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 127631360. Throughput: 0: 1781.1, 1: 1798.0. Samples: 31921680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:16:38,442][51532] Avg episode reward: [(0, '56.130'), (1, '64.580')] -[2023-10-15 17:16:38,499][52866] Updated weights for policy 1, policy_version 62440 (0.0009) -[2023-10-15 17:16:38,513][52833] Updated weights for policy 0, policy_version 62230 (0.0008) -[2023-10-15 17:16:38,873][52866] Updated weights for policy 1, policy_version 62450 (0.0008) -[2023-10-15 17:16:38,888][52833] Updated weights for policy 0, policy_version 62240 (0.0008) -[2023-10-15 17:16:39,239][52866] Updated weights for policy 1, policy_version 62460 (0.0007) -[2023-10-15 17:16:42,792][52833] Updated weights for policy 0, policy_version 62250 (0.0010) -[2023-10-15 17:16:43,039][52866] Updated weights for policy 1, policy_version 62470 (0.0008) -[2023-10-15 17:16:43,166][52833] Updated weights for policy 0, policy_version 62260 (0.0008) -[2023-10-15 17:16:43,408][52866] Updated weights for policy 1, policy_version 62480 (0.0008) -[2023-10-15 17:16:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 127696896. Throughput: 0: 1803.1, 1: 1808.3. Samples: 31943366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:16:43,441][51532] Avg episode reward: [(0, '55.330'), (1, '66.970')] -[2023-10-15 17:16:43,522][52833] Updated weights for policy 0, policy_version 62270 (0.0008) -[2023-10-15 17:16:43,769][52866] Updated weights for policy 1, policy_version 62490 (0.0009) -[2023-10-15 17:16:47,292][52833] Updated weights for policy 0, policy_version 62280 (0.0008) -[2023-10-15 17:16:47,429][52866] Updated weights for policy 1, policy_version 62500 (0.0010) -[2023-10-15 17:16:47,656][52833] Updated weights for policy 0, policy_version 62290 (0.0007) -[2023-10-15 17:16:47,795][52866] Updated weights for policy 1, policy_version 62510 (0.0007) -[2023-10-15 17:16:48,028][52833] Updated weights for policy 0, policy_version 62300 (0.0009) -[2023-10-15 17:16:48,166][52866] Updated weights for policy 1, policy_version 62520 (0.0008) -[2023-10-15 17:16:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 127795200. Throughput: 0: 1783.4, 1: 1796.0. Samples: 31953750. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:16:48,441][51532] Avg episode reward: [(0, '54.080'), (1, '67.460')] -[2023-10-15 17:16:51,799][52833] Updated weights for policy 0, policy_version 62310 (0.0007) -[2023-10-15 17:16:51,936][52866] Updated weights for policy 1, policy_version 62530 (0.0008) -[2023-10-15 17:16:52,162][52833] Updated weights for policy 0, policy_version 62320 (0.0008) -[2023-10-15 17:16:52,293][52866] Updated weights for policy 1, policy_version 62540 (0.0008) -[2023-10-15 17:16:52,534][52833] Updated weights for policy 0, policy_version 62330 (0.0008) -[2023-10-15 17:16:52,663][52866] Updated weights for policy 1, policy_version 62550 (0.0007) -[2023-10-15 17:16:53,038][52866] Updated weights for policy 1, policy_version 62560 (0.0009) -[2023-10-15 17:16:53,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 127893504. Throughput: 0: 1803.5, 1: 1809.2. Samples: 31975810. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:16:53,442][51532] Avg episode reward: [(0, '54.440'), (1, '64.830')] -[2023-10-15 17:16:56,363][52833] Updated weights for policy 0, policy_version 62340 (0.0007) -[2023-10-15 17:16:56,672][52866] Updated weights for policy 1, policy_version 62570 (0.0008) -[2023-10-15 17:16:56,726][52833] Updated weights for policy 0, policy_version 62350 (0.0007) -[2023-10-15 17:16:57,031][52866] Updated weights for policy 1, policy_version 62580 (0.0008) -[2023-10-15 17:16:57,096][52833] Updated weights for policy 0, policy_version 62360 (0.0009) -[2023-10-15 17:16:57,395][52866] Updated weights for policy 1, policy_version 62590 (0.0009) -[2023-10-15 17:16:58,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 127959040. Throughput: 0: 1776.1, 1: 1794.4. Samples: 31995618. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:16:58,442][51532] Avg episode reward: [(0, '54.000'), (1, '65.610')] -[2023-10-15 17:17:00,794][52833] Updated weights for policy 0, policy_version 62370 (0.0007) -[2023-10-15 17:17:01,171][52833] Updated weights for policy 0, policy_version 62380 (0.0008) -[2023-10-15 17:17:01,231][52866] Updated weights for policy 1, policy_version 62600 (0.0008) -[2023-10-15 17:17:01,531][52833] Updated weights for policy 0, policy_version 62390 (0.0008) -[2023-10-15 17:17:01,597][52866] Updated weights for policy 1, policy_version 62610 (0.0008) -[2023-10-15 17:17:01,901][52833] Updated weights for policy 0, policy_version 62400 (0.0008) -[2023-10-15 17:17:01,969][52866] Updated weights for policy 1, policy_version 62620 (0.0007) -[2023-10-15 17:17:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128024576. Throughput: 0: 1801.7, 1: 1802.4. Samples: 32008166. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:17:03,441][51532] Avg episode reward: [(0, '54.250'), (1, '66.930')] -[2023-10-15 17:17:05,631][52833] Updated weights for policy 0, policy_version 62410 (0.0007) -[2023-10-15 17:17:05,811][52866] Updated weights for policy 1, policy_version 62630 (0.0011) -[2023-10-15 17:17:05,998][52833] Updated weights for policy 0, policy_version 62420 (0.0008) -[2023-10-15 17:17:06,179][52866] Updated weights for policy 1, policy_version 62640 (0.0009) -[2023-10-15 17:17:06,370][52833] Updated weights for policy 0, policy_version 62430 (0.0009) -[2023-10-15 17:17:06,551][52866] Updated weights for policy 1, policy_version 62650 (0.0008) -[2023-10-15 17:17:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 128090112. Throughput: 0: 1785.3, 1: 1788.6. Samples: 32027734. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:17:08,441][51532] Avg episode reward: [(0, '57.170'), (1, '63.700')] -[2023-10-15 17:17:10,097][52866] Updated weights for policy 1, policy_version 62660 (0.0008) -[2023-10-15 17:17:10,146][52833] Updated weights for policy 0, policy_version 62440 (0.0008) -[2023-10-15 17:17:10,468][52866] Updated weights for policy 1, policy_version 62670 (0.0008) -[2023-10-15 17:17:10,527][52833] Updated weights for policy 0, policy_version 62450 (0.0008) -[2023-10-15 17:17:10,843][52866] Updated weights for policy 1, policy_version 62680 (0.0008) -[2023-10-15 17:17:10,903][52833] Updated weights for policy 0, policy_version 62460 (0.0009) -[2023-10-15 17:17:13,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 128155648. Throughput: 0: 1784.5, 1: 1791.0. Samples: 32050434. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:17:13,442][51532] Avg episode reward: [(0, '55.280'), (1, '61.460')] -[2023-10-15 17:17:14,538][52866] Updated weights for policy 1, policy_version 62690 (0.0010) -[2023-10-15 17:17:14,600][52833] Updated weights for policy 0, policy_version 62470 (0.0009) -[2023-10-15 17:17:14,894][52866] Updated weights for policy 1, policy_version 62700 (0.0008) -[2023-10-15 17:17:14,970][52833] Updated weights for policy 0, policy_version 62480 (0.0007) -[2023-10-15 17:17:15,256][52866] Updated weights for policy 1, policy_version 62710 (0.0007) -[2023-10-15 17:17:15,327][52833] Updated weights for policy 0, policy_version 62490 (0.0008) -[2023-10-15 17:17:15,625][52866] Updated weights for policy 1, policy_version 62720 (0.0007) -[2023-10-15 17:17:18,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 128221184. Throughput: 0: 1787.5, 1: 1791.6. Samples: 32060424. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:17:18,442][51532] Avg episode reward: [(0, '55.240'), (1, '61.750')] -[2023-10-15 17:17:19,145][52833] Updated weights for policy 0, policy_version 62500 (0.0008) -[2023-10-15 17:17:19,282][52866] Updated weights for policy 1, policy_version 62730 (0.0007) -[2023-10-15 17:17:19,520][52833] Updated weights for policy 0, policy_version 62510 (0.0009) -[2023-10-15 17:17:19,648][52866] Updated weights for policy 1, policy_version 62740 (0.0009) -[2023-10-15 17:17:19,893][52833] Updated weights for policy 0, policy_version 62520 (0.0008) -[2023-10-15 17:17:20,004][52866] Updated weights for policy 1, policy_version 62750 (0.0009) -[2023-10-15 17:17:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 128286720. Throughput: 0: 1781.3, 1: 1799.9. Samples: 32082832. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:17:23,442][51532] Avg episode reward: [(0, '55.710'), (1, '61.060')] -[2023-10-15 17:17:23,701][52833] Updated weights for policy 0, policy_version 62530 (0.0009) -[2023-10-15 17:17:23,761][52866] Updated weights for policy 1, policy_version 62760 (0.0007) -[2023-10-15 17:17:24,061][52833] Updated weights for policy 0, policy_version 62540 (0.0008) -[2023-10-15 17:17:24,120][52866] Updated weights for policy 1, policy_version 62770 (0.0007) -[2023-10-15 17:17:24,429][52833] Updated weights for policy 0, policy_version 62550 (0.0008) -[2023-10-15 17:17:24,479][52866] Updated weights for policy 1, policy_version 62780 (0.0007) -[2023-10-15 17:17:24,799][52833] Updated weights for policy 0, policy_version 62560 (0.0008) -[2023-10-15 17:17:28,370][52866] Updated weights for policy 1, policy_version 62790 (0.0008) -[2023-10-15 17:17:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 128352256. Throughput: 0: 1801.3, 1: 1800.1. Samples: 32105428. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 17:17:28,442][51532] Avg episode reward: [(0, '53.980'), (1, '57.760')] -[2023-10-15 17:17:28,498][52833] Updated weights for policy 0, policy_version 62570 (0.0008) -[2023-10-15 17:17:28,729][52866] Updated weights for policy 1, policy_version 62800 (0.0008) -[2023-10-15 17:17:28,860][52833] Updated weights for policy 0, policy_version 62580 (0.0007) -[2023-10-15 17:17:29,089][52866] Updated weights for policy 1, policy_version 62810 (0.0007) -[2023-10-15 17:17:29,227][52833] Updated weights for policy 0, policy_version 62590 (0.0008) -[2023-10-15 17:17:32,863][52866] Updated weights for policy 1, policy_version 62820 (0.0007) -[2023-10-15 17:17:33,135][52833] Updated weights for policy 0, policy_version 62600 (0.0007) -[2023-10-15 17:17:33,236][52866] Updated weights for policy 1, policy_version 62830 (0.0007) -[2023-10-15 17:17:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 128417792. Throughput: 0: 1790.0, 1: 1796.1. Samples: 32115128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:17:33,442][51532] Avg episode reward: [(0, '56.200'), (1, '58.210')] -[2023-10-15 17:17:33,503][52833] Updated weights for policy 0, policy_version 62610 (0.0008) -[2023-10-15 17:17:33,601][52866] Updated weights for policy 1, policy_version 62840 (0.0007) -[2023-10-15 17:17:33,863][52833] Updated weights for policy 0, policy_version 62620 (0.0007) -[2023-10-15 17:17:37,498][52866] Updated weights for policy 1, policy_version 62850 (0.0007) -[2023-10-15 17:17:37,623][52833] Updated weights for policy 0, policy_version 62630 (0.0009) -[2023-10-15 17:17:37,858][52866] Updated weights for policy 1, policy_version 62860 (0.0009) -[2023-10-15 17:17:37,997][52833] Updated weights for policy 0, policy_version 62640 (0.0007) -[2023-10-15 17:17:38,219][52866] Updated weights for policy 1, policy_version 62870 (0.0007) -[2023-10-15 17:17:38,360][52833] Updated weights for policy 0, policy_version 62650 (0.0007) -[2023-10-15 17:17:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 128483328. Throughput: 0: 1797.1, 1: 1796.9. Samples: 32137542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:17:38,441][51532] Avg episode reward: [(0, '59.820'), (1, '58.820')] -[2023-10-15 17:17:38,588][52866] Updated weights for policy 1, policy_version 62880 (0.0008) -[2023-10-15 17:17:42,029][52833] Updated weights for policy 0, policy_version 62660 (0.0008) -[2023-10-15 17:17:42,278][52866] Updated weights for policy 1, policy_version 62890 (0.0010) -[2023-10-15 17:17:42,397][52833] Updated weights for policy 0, policy_version 62670 (0.0008) -[2023-10-15 17:17:42,655][52866] Updated weights for policy 1, policy_version 62900 (0.0010) -[2023-10-15 17:17:42,771][52833] Updated weights for policy 0, policy_version 62680 (0.0007) -[2023-10-15 17:17:43,019][52866] Updated weights for policy 1, policy_version 62910 (0.0009) -[2023-10-15 17:17:43,441][51532] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 128614400. Throughput: 0: 1800.1, 1: 1804.8. Samples: 32157838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:17:43,441][51532] Avg episode reward: [(0, '59.380'), (1, '59.340')] -[2023-10-15 17:17:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000062912_64421888.pth... -[2023-10-15 17:17:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000062688_64192512.pth... -[2023-10-15 17:17:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000061216_62685184.pth -[2023-10-15 17:17:43,492][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000060992_62455808.pth -[2023-10-15 17:17:46,574][52833] Updated weights for policy 0, policy_version 62690 (0.0008) -[2023-10-15 17:17:46,717][52866] Updated weights for policy 1, policy_version 62920 (0.0009) -[2023-10-15 17:17:46,946][52833] Updated weights for policy 0, policy_version 62700 (0.0008) -[2023-10-15 17:17:47,080][52866] Updated weights for policy 1, policy_version 62930 (0.0009) -[2023-10-15 17:17:47,314][52833] Updated weights for policy 0, policy_version 62710 (0.0007) -[2023-10-15 17:17:47,443][52866] Updated weights for policy 1, policy_version 62940 (0.0008) -[2023-10-15 17:17:47,679][52833] Updated weights for policy 0, policy_version 62720 (0.0008) -[2023-10-15 17:17:48,441][51532] Fps is (10 sec: 19660.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 128679936. Throughput: 0: 1793.1, 1: 1802.7. Samples: 32169976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:17:48,442][51532] Avg episode reward: [(0, '59.420'), (1, '58.380')] -[2023-10-15 17:17:51,307][52866] Updated weights for policy 1, policy_version 62950 (0.0007) -[2023-10-15 17:17:51,353][52833] Updated weights for policy 0, policy_version 62730 (0.0008) -[2023-10-15 17:17:51,665][52866] Updated weights for policy 1, policy_version 62960 (0.0008) -[2023-10-15 17:17:51,705][52833] Updated weights for policy 0, policy_version 62740 (0.0008) -[2023-10-15 17:17:52,028][52866] Updated weights for policy 1, policy_version 62970 (0.0009) -[2023-10-15 17:17:52,069][52833] Updated weights for policy 0, policy_version 62750 (0.0008) -[2023-10-15 17:17:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128745472. Throughput: 0: 1801.1, 1: 1810.8. Samples: 32190270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:17:53,441][51532] Avg episode reward: [(0, '55.990'), (1, '59.400')] -[2023-10-15 17:17:55,695][52866] Updated weights for policy 1, policy_version 62980 (0.0008) -[2023-10-15 17:17:55,836][52833] Updated weights for policy 0, policy_version 62760 (0.0008) -[2023-10-15 17:17:56,057][52866] Updated weights for policy 1, policy_version 62990 (0.0008) -[2023-10-15 17:17:56,201][52833] Updated weights for policy 0, policy_version 62770 (0.0009) -[2023-10-15 17:17:56,424][52866] Updated weights for policy 1, policy_version 63000 (0.0010) -[2023-10-15 17:17:56,562][52833] Updated weights for policy 0, policy_version 62780 (0.0008) -[2023-10-15 17:17:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128811008. Throughput: 0: 1794.9, 1: 1797.2. Samples: 32212078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:17:58,441][51532] Avg episode reward: [(0, '53.850'), (1, '60.600')] -[2023-10-15 17:18:00,142][52866] Updated weights for policy 1, policy_version 63010 (0.0010) -[2023-10-15 17:18:00,404][52833] Updated weights for policy 0, policy_version 62790 (0.0009) -[2023-10-15 17:18:00,506][52866] Updated weights for policy 1, policy_version 63020 (0.0008) -[2023-10-15 17:18:00,771][52833] Updated weights for policy 0, policy_version 62800 (0.0008) -[2023-10-15 17:18:00,867][52866] Updated weights for policy 1, policy_version 63030 (0.0007) -[2023-10-15 17:18:01,137][52833] Updated weights for policy 0, policy_version 62810 (0.0008) -[2023-10-15 17:18:01,236][52866] Updated weights for policy 1, policy_version 63040 (0.0009) -[2023-10-15 17:18:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 128876544. Throughput: 0: 1803.6, 1: 1806.5. Samples: 32222880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:18:03,442][51532] Avg episode reward: [(0, '54.910'), (1, '63.160')] -[2023-10-15 17:18:04,822][52833] Updated weights for policy 0, policy_version 62820 (0.0008) -[2023-10-15 17:18:04,975][52866] Updated weights for policy 1, policy_version 63050 (0.0007) -[2023-10-15 17:18:05,187][52833] Updated weights for policy 0, policy_version 62830 (0.0008) -[2023-10-15 17:18:05,341][52866] Updated weights for policy 1, policy_version 63060 (0.0007) -[2023-10-15 17:18:05,555][52833] Updated weights for policy 0, policy_version 62840 (0.0007) -[2023-10-15 17:18:05,707][52866] Updated weights for policy 1, policy_version 63070 (0.0007) -[2023-10-15 17:18:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 128942080. Throughput: 0: 1790.5, 1: 1792.8. Samples: 32244078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:18:08,441][51532] Avg episode reward: [(0, '57.880'), (1, '60.810')] -[2023-10-15 17:18:09,344][52833] Updated weights for policy 0, policy_version 62850 (0.0007) -[2023-10-15 17:18:09,580][52866] Updated weights for policy 1, policy_version 63080 (0.0007) -[2023-10-15 17:18:09,716][52833] Updated weights for policy 0, policy_version 62860 (0.0009) -[2023-10-15 17:18:09,959][52866] Updated weights for policy 1, policy_version 63090 (0.0008) -[2023-10-15 17:18:10,082][52833] Updated weights for policy 0, policy_version 62870 (0.0009) -[2023-10-15 17:18:10,321][52866] Updated weights for policy 1, policy_version 63100 (0.0007) -[2023-10-15 17:18:10,448][52833] Updated weights for policy 0, policy_version 62880 (0.0008) -[2023-10-15 17:18:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 129007616. Throughput: 0: 1789.0, 1: 1789.7. Samples: 32266472. Policy #0 lag: (min: 18.0, avg: 22.6, max: 50.0) -[2023-10-15 17:18:13,442][51532] Avg episode reward: [(0, '59.200'), (1, '59.180')] -[2023-10-15 17:18:13,980][52866] Updated weights for policy 1, policy_version 63110 (0.0008) -[2023-10-15 17:18:14,107][52833] Updated weights for policy 0, policy_version 62890 (0.0007) -[2023-10-15 17:18:14,341][52866] Updated weights for policy 1, policy_version 63120 (0.0009) -[2023-10-15 17:18:14,476][52833] Updated weights for policy 0, policy_version 62900 (0.0010) -[2023-10-15 17:18:14,709][52866] Updated weights for policy 1, policy_version 63130 (0.0008) -[2023-10-15 17:18:14,850][52833] Updated weights for policy 0, policy_version 62910 (0.0009) -[2023-10-15 17:18:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 129073152. Throughput: 0: 1790.0, 1: 1788.9. Samples: 32276176. Policy #0 lag: (min: 18.0, avg: 22.6, max: 50.0) -[2023-10-15 17:18:18,441][51532] Avg episode reward: [(0, '59.180'), (1, '58.270')] -[2023-10-15 17:18:18,629][52866] Updated weights for policy 1, policy_version 63140 (0.0007) -[2023-10-15 17:18:18,700][52833] Updated weights for policy 0, policy_version 62920 (0.0010) -[2023-10-15 17:18:19,004][52866] Updated weights for policy 1, policy_version 63150 (0.0007) -[2023-10-15 17:18:19,073][52833] Updated weights for policy 0, policy_version 62930 (0.0009) -[2023-10-15 17:18:19,369][52866] Updated weights for policy 1, policy_version 63160 (0.0007) -[2023-10-15 17:18:19,445][52833] Updated weights for policy 0, policy_version 62940 (0.0008) -[2023-10-15 17:18:23,097][52866] Updated weights for policy 1, policy_version 63170 (0.0008) -[2023-10-15 17:18:23,150][52833] Updated weights for policy 0, policy_version 62950 (0.0007) -[2023-10-15 17:18:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 129138688. Throughput: 0: 1787.5, 1: 1785.3. Samples: 32298318. Policy #0 lag: (min: 18.0, avg: 22.6, max: 50.0) -[2023-10-15 17:18:23,443][51532] Avg episode reward: [(0, '57.540'), (1, '58.260')] -[2023-10-15 17:18:23,465][52866] Updated weights for policy 1, policy_version 63180 (0.0008) -[2023-10-15 17:18:23,514][52833] Updated weights for policy 0, policy_version 62960 (0.0009) -[2023-10-15 17:18:23,825][52866] Updated weights for policy 1, policy_version 63190 (0.0008) -[2023-10-15 17:18:23,888][52833] Updated weights for policy 0, policy_version 62970 (0.0008) -[2023-10-15 17:18:24,191][52866] Updated weights for policy 1, policy_version 63200 (0.0009) -[2023-10-15 17:18:27,681][52833] Updated weights for policy 0, policy_version 62980 (0.0008) -[2023-10-15 17:18:28,019][52866] Updated weights for policy 1, policy_version 63210 (0.0008) -[2023-10-15 17:18:28,054][52833] Updated weights for policy 0, policy_version 62990 (0.0009) -[2023-10-15 17:18:28,391][52866] Updated weights for policy 1, policy_version 63220 (0.0010) -[2023-10-15 17:18:28,414][52833] Updated weights for policy 0, policy_version 63000 (0.0007) -[2023-10-15 17:18:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 129204224. Throughput: 0: 1802.4, 1: 1798.1. Samples: 32319862. Policy #0 lag: (min: 18.0, avg: 22.6, max: 50.0) -[2023-10-15 17:18:28,442][51532] Avg episode reward: [(0, '56.830'), (1, '60.650')] -[2023-10-15 17:18:28,753][52866] Updated weights for policy 1, policy_version 63230 (0.0008) -[2023-10-15 17:18:32,306][52833] Updated weights for policy 0, policy_version 63010 (0.0007) -[2023-10-15 17:18:32,569][52866] Updated weights for policy 1, policy_version 63240 (0.0008) -[2023-10-15 17:18:32,674][52833] Updated weights for policy 0, policy_version 63020 (0.0007) -[2023-10-15 17:18:32,931][52866] Updated weights for policy 1, policy_version 63250 (0.0007) -[2023-10-15 17:18:33,044][52833] Updated weights for policy 0, policy_version 63030 (0.0008) -[2023-10-15 17:18:33,303][52866] Updated weights for policy 1, policy_version 63260 (0.0008) -[2023-10-15 17:18:33,406][52833] Updated weights for policy 0, policy_version 63040 (0.0008) -[2023-10-15 17:18:33,441][51532] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 129302528. Throughput: 0: 1784.6, 1: 1779.1. Samples: 32330342. Policy #0 lag: (min: 18.0, avg: 22.6, max: 50.0) -[2023-10-15 17:18:33,441][51532] Avg episode reward: [(0, '56.140'), (1, '63.570')] -[2023-10-15 17:18:37,088][52866] Updated weights for policy 1, policy_version 63270 (0.0008) -[2023-10-15 17:18:37,166][52833] Updated weights for policy 0, policy_version 63050 (0.0007) -[2023-10-15 17:18:37,460][52866] Updated weights for policy 1, policy_version 63280 (0.0007) -[2023-10-15 17:18:37,527][52833] Updated weights for policy 0, policy_version 63060 (0.0009) -[2023-10-15 17:18:37,817][52866] Updated weights for policy 1, policy_version 63290 (0.0009) -[2023-10-15 17:18:37,894][52833] Updated weights for policy 0, policy_version 63070 (0.0007) -[2023-10-15 17:18:38,441][51532] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 129400832. Throughput: 0: 1799.2, 1: 1800.5. Samples: 32352260. Policy #0 lag: (min: 18.0, avg: 22.6, max: 50.0) -[2023-10-15 17:18:38,442][51532] Avg episode reward: [(0, '53.210'), (1, '60.400')] -[2023-10-15 17:18:41,590][52866] Updated weights for policy 1, policy_version 63300 (0.0009) -[2023-10-15 17:18:41,771][52833] Updated weights for policy 0, policy_version 63080 (0.0008) -[2023-10-15 17:18:41,955][52866] Updated weights for policy 1, policy_version 63310 (0.0009) -[2023-10-15 17:18:42,146][52833] Updated weights for policy 0, policy_version 63090 (0.0007) -[2023-10-15 17:18:42,324][52866] Updated weights for policy 1, policy_version 63320 (0.0007) -[2023-10-15 17:18:42,517][52833] Updated weights for policy 0, policy_version 63100 (0.0007) -[2023-10-15 17:18:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 129466368. Throughput: 0: 1770.4, 1: 1777.5. Samples: 32371734. Policy #0 lag: (min: 18.0, avg: 22.6, max: 50.0) -[2023-10-15 17:18:43,442][51532] Avg episode reward: [(0, '54.680'), (1, '61.560')] -[2023-10-15 17:18:46,100][52833] Updated weights for policy 0, policy_version 63110 (0.0008) -[2023-10-15 17:18:46,172][52866] Updated weights for policy 1, policy_version 63330 (0.0007) -[2023-10-15 17:18:46,479][52833] Updated weights for policy 0, policy_version 63120 (0.0008) -[2023-10-15 17:18:46,535][52866] Updated weights for policy 1, policy_version 63340 (0.0008) -[2023-10-15 17:18:46,842][52833] Updated weights for policy 0, policy_version 63130 (0.0008) -[2023-10-15 17:18:46,903][52866] Updated weights for policy 1, policy_version 63350 (0.0007) -[2023-10-15 17:18:47,265][52866] Updated weights for policy 1, policy_version 63360 (0.0007) -[2023-10-15 17:18:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 129531904. Throughput: 0: 1794.7, 1: 1797.6. Samples: 32384536. Policy #0 lag: (min: 18.0, avg: 22.6, max: 50.0) -[2023-10-15 17:18:48,441][51532] Avg episode reward: [(0, '55.000'), (1, '60.820')] -[2023-10-15 17:18:50,658][52833] Updated weights for policy 0, policy_version 63140 (0.0007) -[2023-10-15 17:18:50,971][52866] Updated weights for policy 1, policy_version 63370 (0.0007) -[2023-10-15 17:18:51,024][52833] Updated weights for policy 0, policy_version 63150 (0.0009) -[2023-10-15 17:18:51,335][52866] Updated weights for policy 1, policy_version 63380 (0.0008) -[2023-10-15 17:18:51,398][52833] Updated weights for policy 0, policy_version 63160 (0.0008) -[2023-10-15 17:18:51,704][52866] Updated weights for policy 1, policy_version 63390 (0.0008) -[2023-10-15 17:18:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 129597440. Throughput: 0: 1779.6, 1: 1776.1. Samples: 32404086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:18:53,442][51532] Avg episode reward: [(0, '52.700'), (1, '59.010')] -[2023-10-15 17:18:55,040][52833] Updated weights for policy 0, policy_version 63170 (0.0009) -[2023-10-15 17:18:55,411][52833] Updated weights for policy 0, policy_version 63180 (0.0008) -[2023-10-15 17:18:55,464][52866] Updated weights for policy 1, policy_version 63400 (0.0008) -[2023-10-15 17:18:55,781][52833] Updated weights for policy 0, policy_version 63190 (0.0007) -[2023-10-15 17:18:55,833][52866] Updated weights for policy 1, policy_version 63410 (0.0008) -[2023-10-15 17:18:56,147][52833] Updated weights for policy 0, policy_version 63200 (0.0007) -[2023-10-15 17:18:56,194][52866] Updated weights for policy 1, policy_version 63420 (0.0010) -[2023-10-15 17:18:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 129662976. Throughput: 0: 1780.7, 1: 1776.4. Samples: 32426542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:18:58,442][51532] Avg episode reward: [(0, '50.020'), (1, '61.830')] -[2023-10-15 17:18:59,886][52833] Updated weights for policy 0, policy_version 63210 (0.0007) -[2023-10-15 17:19:00,019][52866] Updated weights for policy 1, policy_version 63430 (0.0011) -[2023-10-15 17:19:00,240][52833] Updated weights for policy 0, policy_version 63220 (0.0008) -[2023-10-15 17:19:00,381][52866] Updated weights for policy 1, policy_version 63440 (0.0008) -[2023-10-15 17:19:00,611][52833] Updated weights for policy 0, policy_version 63230 (0.0007) -[2023-10-15 17:19:00,748][52866] Updated weights for policy 1, policy_version 63450 (0.0009) -[2023-10-15 17:19:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 129728512. Throughput: 0: 1784.5, 1: 1775.1. Samples: 32436358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:03,442][51532] Avg episode reward: [(0, '53.710'), (1, '60.490')] -[2023-10-15 17:19:04,335][52833] Updated weights for policy 0, policy_version 63240 (0.0007) -[2023-10-15 17:19:04,510][52866] Updated weights for policy 1, policy_version 63460 (0.0007) -[2023-10-15 17:19:04,705][52833] Updated weights for policy 0, policy_version 63250 (0.0009) -[2023-10-15 17:19:04,879][52866] Updated weights for policy 1, policy_version 63470 (0.0008) -[2023-10-15 17:19:05,082][52833] Updated weights for policy 0, policy_version 63260 (0.0008) -[2023-10-15 17:19:05,243][52866] Updated weights for policy 1, policy_version 63480 (0.0008) -[2023-10-15 17:19:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 129794048. Throughput: 0: 1787.6, 1: 1778.1. Samples: 32458774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:08,442][51532] Avg episode reward: [(0, '54.510'), (1, '61.810')] -[2023-10-15 17:19:08,982][52833] Updated weights for policy 0, policy_version 63270 (0.0007) -[2023-10-15 17:19:09,161][52866] Updated weights for policy 1, policy_version 63490 (0.0009) -[2023-10-15 17:19:09,351][52833] Updated weights for policy 0, policy_version 63280 (0.0008) -[2023-10-15 17:19:09,522][52866] Updated weights for policy 1, policy_version 63500 (0.0010) -[2023-10-15 17:19:09,716][52833] Updated weights for policy 0, policy_version 63290 (0.0008) -[2023-10-15 17:19:09,894][52866] Updated weights for policy 1, policy_version 63510 (0.0008) -[2023-10-15 17:19:10,263][52866] Updated weights for policy 1, policy_version 63520 (0.0011) -[2023-10-15 17:19:13,315][52833] Updated weights for policy 0, policy_version 63300 (0.0009) -[2023-10-15 17:19:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 129859584. Throughput: 0: 1797.2, 1: 1783.7. Samples: 32481004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:13,442][51532] Avg episode reward: [(0, '54.290'), (1, '63.350')] -[2023-10-15 17:19:13,680][52833] Updated weights for policy 0, policy_version 63310 (0.0010) -[2023-10-15 17:19:14,057][52833] Updated weights for policy 0, policy_version 63320 (0.0009) -[2023-10-15 17:19:14,141][52866] Updated weights for policy 1, policy_version 63530 (0.0007) -[2023-10-15 17:19:14,511][52866] Updated weights for policy 1, policy_version 63540 (0.0009) -[2023-10-15 17:19:14,874][52866] Updated weights for policy 1, policy_version 63550 (0.0010) -[2023-10-15 17:19:17,822][52833] Updated weights for policy 0, policy_version 63330 (0.0007) -[2023-10-15 17:19:18,178][52833] Updated weights for policy 0, policy_version 63340 (0.0007) -[2023-10-15 17:19:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 129925120. Throughput: 0: 1792.7, 1: 1773.5. Samples: 32490818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:18,441][51532] Avg episode reward: [(0, '54.600'), (1, '62.410')] -[2023-10-15 17:19:18,551][52833] Updated weights for policy 0, policy_version 63350 (0.0008) -[2023-10-15 17:19:18,678][52866] Updated weights for policy 1, policy_version 63560 (0.0008) -[2023-10-15 17:19:18,925][52833] Updated weights for policy 0, policy_version 63360 (0.0007) -[2023-10-15 17:19:19,048][52866] Updated weights for policy 1, policy_version 63570 (0.0007) -[2023-10-15 17:19:19,413][52866] Updated weights for policy 1, policy_version 63580 (0.0008) -[2023-10-15 17:19:22,894][52833] Updated weights for policy 0, policy_version 63370 (0.0009) -[2023-10-15 17:19:23,153][52866] Updated weights for policy 1, policy_version 63590 (0.0008) -[2023-10-15 17:19:23,252][52833] Updated weights for policy 0, policy_version 63380 (0.0009) -[2023-10-15 17:19:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.6, 300 sec: 14218.0). Total num frames: 129990656. Throughput: 0: 1794.0, 1: 1781.2. Samples: 32513140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:23,441][51532] Avg episode reward: [(0, '56.510'), (1, '59.580')] -[2023-10-15 17:19:23,513][52866] Updated weights for policy 1, policy_version 63600 (0.0007) -[2023-10-15 17:19:23,627][52833] Updated weights for policy 0, policy_version 63390 (0.0009) -[2023-10-15 17:19:23,879][52866] Updated weights for policy 1, policy_version 63610 (0.0008) -[2023-10-15 17:19:27,449][52833] Updated weights for policy 0, policy_version 63400 (0.0009) -[2023-10-15 17:19:27,588][52866] Updated weights for policy 1, policy_version 63620 (0.0008) -[2023-10-15 17:19:27,827][52833] Updated weights for policy 0, policy_version 63410 (0.0008) -[2023-10-15 17:19:27,952][52866] Updated weights for policy 1, policy_version 63630 (0.0007) -[2023-10-15 17:19:28,185][52833] Updated weights for policy 0, policy_version 63420 (0.0007) -[2023-10-15 17:19:28,306][52866] Updated weights for policy 1, policy_version 63640 (0.0009) -[2023-10-15 17:19:28,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 130088960. Throughput: 0: 1809.5, 1: 1800.8. Samples: 32534196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:28,441][51532] Avg episode reward: [(0, '60.290'), (1, '59.990')] -[2023-10-15 17:19:31,787][52833] Updated weights for policy 0, policy_version 63430 (0.0008) -[2023-10-15 17:19:32,001][52866] Updated weights for policy 1, policy_version 63650 (0.0010) -[2023-10-15 17:19:32,152][52833] Updated weights for policy 0, policy_version 63440 (0.0007) -[2023-10-15 17:19:32,368][52866] Updated weights for policy 1, policy_version 63660 (0.0008) -[2023-10-15 17:19:32,518][52833] Updated weights for policy 0, policy_version 63450 (0.0007) -[2023-10-15 17:19:32,727][52866] Updated weights for policy 1, policy_version 63670 (0.0007) -[2023-10-15 17:19:33,100][52866] Updated weights for policy 1, policy_version 63680 (0.0008) -[2023-10-15 17:19:33,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 130187264. Throughput: 0: 1791.6, 1: 1781.5. Samples: 32545326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:33,441][51532] Avg episode reward: [(0, '61.220'), (1, '60.130')] -[2023-10-15 17:19:36,403][52833] Updated weights for policy 0, policy_version 63460 (0.0008) -[2023-10-15 17:19:36,772][52866] Updated weights for policy 1, policy_version 63690 (0.0007) -[2023-10-15 17:19:36,780][52833] Updated weights for policy 0, policy_version 63470 (0.0008) -[2023-10-15 17:19:37,134][52833] Updated weights for policy 0, policy_version 63480 (0.0010) -[2023-10-15 17:19:37,142][52866] Updated weights for policy 1, policy_version 63700 (0.0007) -[2023-10-15 17:19:37,511][52866] Updated weights for policy 1, policy_version 63710 (0.0008) -[2023-10-15 17:19:38,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 130252800. Throughput: 0: 1802.9, 1: 1798.6. Samples: 32566152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:38,442][51532] Avg episode reward: [(0, '61.850'), (1, '61.560')] -[2023-10-15 17:19:40,935][52833] Updated weights for policy 0, policy_version 63490 (0.0010) -[2023-10-15 17:19:41,296][52833] Updated weights for policy 0, policy_version 63500 (0.0009) -[2023-10-15 17:19:41,364][52866] Updated weights for policy 1, policy_version 63720 (0.0008) -[2023-10-15 17:19:41,672][52833] Updated weights for policy 0, policy_version 63510 (0.0010) -[2023-10-15 17:19:41,740][52866] Updated weights for policy 1, policy_version 63730 (0.0009) -[2023-10-15 17:19:42,036][52833] Updated weights for policy 0, policy_version 63520 (0.0008) -[2023-10-15 17:19:42,106][52866] Updated weights for policy 1, policy_version 63740 (0.0008) -[2023-10-15 17:19:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 130318336. Throughput: 0: 1787.2, 1: 1782.2. Samples: 32587162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:43,442][51532] Avg episode reward: [(0, '61.730'), (1, '59.150')] -[2023-10-15 17:19:43,455][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000063520_65044480.pth... -[2023-10-15 17:19:43,456][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000063744_65273856.pth... -[2023-10-15 17:19:43,490][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000061856_63340544.pth -[2023-10-15 17:19:43,490][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000062048_63537152.pth -[2023-10-15 17:19:45,639][52866] Updated weights for policy 1, policy_version 63750 (0.0008) -[2023-10-15 17:19:45,823][52833] Updated weights for policy 0, policy_version 63530 (0.0007) -[2023-10-15 17:19:45,998][52866] Updated weights for policy 1, policy_version 63760 (0.0007) -[2023-10-15 17:19:46,181][52833] Updated weights for policy 0, policy_version 63540 (0.0009) -[2023-10-15 17:19:46,369][52866] Updated weights for policy 1, policy_version 63770 (0.0008) -[2023-10-15 17:19:46,554][52833] Updated weights for policy 0, policy_version 63550 (0.0010) -[2023-10-15 17:19:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 130383872. Throughput: 0: 1806.8, 1: 1803.2. Samples: 32598810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:48,441][51532] Avg episode reward: [(0, '61.680'), (1, '61.710')] -[2023-10-15 17:19:50,247][52866] Updated weights for policy 1, policy_version 63780 (0.0008) -[2023-10-15 17:19:50,309][52833] Updated weights for policy 0, policy_version 63560 (0.0007) -[2023-10-15 17:19:50,614][52866] Updated weights for policy 1, policy_version 63790 (0.0009) -[2023-10-15 17:19:50,671][52833] Updated weights for policy 0, policy_version 63570 (0.0007) -[2023-10-15 17:19:50,979][52866] Updated weights for policy 1, policy_version 63800 (0.0007) -[2023-10-15 17:19:51,042][52833] Updated weights for policy 0, policy_version 63580 (0.0008) -[2023-10-15 17:19:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 130449408. Throughput: 0: 1783.4, 1: 1786.2. Samples: 32619408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:53,442][51532] Avg episode reward: [(0, '62.180'), (1, '61.000')] -[2023-10-15 17:19:54,861][52833] Updated weights for policy 0, policy_version 63590 (0.0008) -[2023-10-15 17:19:54,893][52866] Updated weights for policy 1, policy_version 63810 (0.0009) -[2023-10-15 17:19:55,216][52833] Updated weights for policy 0, policy_version 63600 (0.0009) -[2023-10-15 17:19:55,262][52866] Updated weights for policy 1, policy_version 63820 (0.0008) -[2023-10-15 17:19:55,596][52833] Updated weights for policy 0, policy_version 63610 (0.0009) -[2023-10-15 17:19:55,630][52866] Updated weights for policy 1, policy_version 63830 (0.0007) -[2023-10-15 17:19:55,995][52866] Updated weights for policy 1, policy_version 63840 (0.0007) -[2023-10-15 17:19:58,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 130514944. Throughput: 0: 1779.5, 1: 1794.1. Samples: 32641816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:19:58,442][51532] Avg episode reward: [(0, '62.100'), (1, '59.650')] -[2023-10-15 17:19:59,283][52833] Updated weights for policy 0, policy_version 63620 (0.0010) -[2023-10-15 17:19:59,658][52833] Updated weights for policy 0, policy_version 63630 (0.0009) -[2023-10-15 17:19:59,726][52866] Updated weights for policy 1, policy_version 63850 (0.0009) -[2023-10-15 17:20:00,026][52833] Updated weights for policy 0, policy_version 63640 (0.0008) -[2023-10-15 17:20:00,087][52866] Updated weights for policy 1, policy_version 63860 (0.0009) -[2023-10-15 17:20:00,452][52866] Updated weights for policy 1, policy_version 63870 (0.0009) -[2023-10-15 17:20:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 130580480. Throughput: 0: 1777.7, 1: 1794.6. Samples: 32651574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:20:03,442][51532] Avg episode reward: [(0, '59.420'), (1, '61.560')] -[2023-10-15 17:20:03,808][52833] Updated weights for policy 0, policy_version 63650 (0.0008) -[2023-10-15 17:20:04,076][52866] Updated weights for policy 1, policy_version 63880 (0.0007) -[2023-10-15 17:20:04,171][52833] Updated weights for policy 0, policy_version 63660 (0.0008) -[2023-10-15 17:20:04,451][52866] Updated weights for policy 1, policy_version 63890 (0.0008) -[2023-10-15 17:20:04,548][52833] Updated weights for policy 0, policy_version 63670 (0.0007) -[2023-10-15 17:20:04,809][52866] Updated weights for policy 1, policy_version 63900 (0.0008) -[2023-10-15 17:20:04,912][52833] Updated weights for policy 0, policy_version 63680 (0.0009) -[2023-10-15 17:20:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 130646016. Throughput: 0: 1782.2, 1: 1792.3. Samples: 32673994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:20:08,441][51532] Avg episode reward: [(0, '58.650'), (1, '61.810')] -[2023-10-15 17:20:08,621][52866] Updated weights for policy 1, policy_version 63910 (0.0009) -[2023-10-15 17:20:08,697][52833] Updated weights for policy 0, policy_version 63690 (0.0008) -[2023-10-15 17:20:08,987][52866] Updated weights for policy 1, policy_version 63920 (0.0009) -[2023-10-15 17:20:09,066][52833] Updated weights for policy 0, policy_version 63700 (0.0008) -[2023-10-15 17:20:09,368][52866] Updated weights for policy 1, policy_version 63930 (0.0008) -[2023-10-15 17:20:09,423][52833] Updated weights for policy 0, policy_version 63710 (0.0007) -[2023-10-15 17:20:13,089][52866] Updated weights for policy 1, policy_version 63940 (0.0009) -[2023-10-15 17:20:13,286][52833] Updated weights for policy 0, policy_version 63720 (0.0008) -[2023-10-15 17:20:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 130711552. Throughput: 0: 1801.6, 1: 1804.5. Samples: 32696474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:20:13,441][51532] Avg episode reward: [(0, '57.460'), (1, '60.610')] -[2023-10-15 17:20:13,451][52866] Updated weights for policy 1, policy_version 63950 (0.0008) -[2023-10-15 17:20:13,653][52833] Updated weights for policy 0, policy_version 63730 (0.0008) -[2023-10-15 17:20:13,826][52866] Updated weights for policy 1, policy_version 63960 (0.0009) -[2023-10-15 17:20:14,017][52833] Updated weights for policy 0, policy_version 63740 (0.0008) -[2023-10-15 17:20:17,535][52866] Updated weights for policy 1, policy_version 63970 (0.0007) -[2023-10-15 17:20:17,786][52833] Updated weights for policy 0, policy_version 63750 (0.0008) -[2023-10-15 17:20:17,908][52866] Updated weights for policy 1, policy_version 63980 (0.0007) -[2023-10-15 17:20:18,157][52833] Updated weights for policy 0, policy_version 63760 (0.0009) -[2023-10-15 17:20:18,265][52866] Updated weights for policy 1, policy_version 63990 (0.0008) -[2023-10-15 17:20:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 130777088. Throughput: 0: 1778.9, 1: 1795.0. Samples: 32706152. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:20:18,441][51532] Avg episode reward: [(0, '59.670'), (1, '65.410')] -[2023-10-15 17:20:18,522][52833] Updated weights for policy 0, policy_version 63770 (0.0009) -[2023-10-15 17:20:18,634][52866] Updated weights for policy 1, policy_version 64000 (0.0007) -[2023-10-15 17:20:22,292][52833] Updated weights for policy 0, policy_version 63780 (0.0009) -[2023-10-15 17:20:22,491][52866] Updated weights for policy 1, policy_version 64010 (0.0009) -[2023-10-15 17:20:22,661][52833] Updated weights for policy 0, policy_version 63790 (0.0009) -[2023-10-15 17:20:22,860][52866] Updated weights for policy 1, policy_version 64020 (0.0007) -[2023-10-15 17:20:23,030][52833] Updated weights for policy 0, policy_version 63800 (0.0008) -[2023-10-15 17:20:23,221][52866] Updated weights for policy 1, policy_version 64030 (0.0007) -[2023-10-15 17:20:23,441][51532] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 130908160. Throughput: 0: 1797.0, 1: 1810.5. Samples: 32728492. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:20:23,442][51532] Avg episode reward: [(0, '54.130'), (1, '64.350')] -[2023-10-15 17:20:26,858][52833] Updated weights for policy 0, policy_version 63810 (0.0008) -[2023-10-15 17:20:27,051][52866] Updated weights for policy 1, policy_version 64040 (0.0009) -[2023-10-15 17:20:27,224][52833] Updated weights for policy 0, policy_version 63820 (0.0008) -[2023-10-15 17:20:27,418][52866] Updated weights for policy 1, policy_version 64050 (0.0008) -[2023-10-15 17:20:27,596][52833] Updated weights for policy 0, policy_version 63830 (0.0008) -[2023-10-15 17:20:27,784][52866] Updated weights for policy 1, policy_version 64060 (0.0009) -[2023-10-15 17:20:27,961][52833] Updated weights for policy 0, policy_version 63840 (0.0007) -[2023-10-15 17:20:28,441][51532] Fps is (10 sec: 19660.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 130973696. Throughput: 0: 1776.6, 1: 1795.0. Samples: 32747884. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:20:28,442][51532] Avg episode reward: [(0, '56.330'), (1, '63.910')] -[2023-10-15 17:20:31,524][52866] Updated weights for policy 1, policy_version 64070 (0.0008) -[2023-10-15 17:20:31,742][52833] Updated weights for policy 0, policy_version 63850 (0.0008) -[2023-10-15 17:20:31,890][52866] Updated weights for policy 1, policy_version 64080 (0.0007) -[2023-10-15 17:20:32,099][52833] Updated weights for policy 0, policy_version 63860 (0.0007) -[2023-10-15 17:20:32,253][52866] Updated weights for policy 1, policy_version 64090 (0.0007) -[2023-10-15 17:20:32,467][52833] Updated weights for policy 0, policy_version 63870 (0.0008) -[2023-10-15 17:20:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 131039232. Throughput: 0: 1783.1, 1: 1807.3. Samples: 32760378. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:20:33,442][51532] Avg episode reward: [(0, '57.710'), (1, '63.220')] -[2023-10-15 17:20:36,123][52866] Updated weights for policy 1, policy_version 64100 (0.0008) -[2023-10-15 17:20:36,367][52833] Updated weights for policy 0, policy_version 63880 (0.0008) -[2023-10-15 17:20:36,487][52866] Updated weights for policy 1, policy_version 64110 (0.0009) -[2023-10-15 17:20:36,730][52833] Updated weights for policy 0, policy_version 63890 (0.0010) -[2023-10-15 17:20:36,852][52866] Updated weights for policy 1, policy_version 64120 (0.0008) -[2023-10-15 17:20:37,091][52833] Updated weights for policy 0, policy_version 63900 (0.0007) -[2023-10-15 17:20:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 131104768. Throughput: 0: 1781.1, 1: 1797.7. Samples: 32780454. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:20:38,442][51532] Avg episode reward: [(0, '58.300'), (1, '62.370')] -[2023-10-15 17:20:40,601][52833] Updated weights for policy 0, policy_version 63910 (0.0008) -[2023-10-15 17:20:40,714][52866] Updated weights for policy 1, policy_version 64130 (0.0009) -[2023-10-15 17:20:40,968][52833] Updated weights for policy 0, policy_version 63920 (0.0008) -[2023-10-15 17:20:41,080][52866] Updated weights for policy 1, policy_version 64140 (0.0009) -[2023-10-15 17:20:41,341][52833] Updated weights for policy 0, policy_version 63930 (0.0008) -[2023-10-15 17:20:41,446][52866] Updated weights for policy 1, policy_version 64150 (0.0009) -[2023-10-15 17:20:41,805][52866] Updated weights for policy 1, policy_version 64160 (0.0008) -[2023-10-15 17:20:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 131170304. Throughput: 0: 1778.0, 1: 1786.6. Samples: 32802226. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:20:43,442][51532] Avg episode reward: [(0, '58.300'), (1, '64.020')] -[2023-10-15 17:20:45,101][52833] Updated weights for policy 0, policy_version 63940 (0.0007) -[2023-10-15 17:20:45,468][52833] Updated weights for policy 0, policy_version 63950 (0.0007) -[2023-10-15 17:20:45,487][52866] Updated weights for policy 1, policy_version 64170 (0.0007) -[2023-10-15 17:20:45,835][52833] Updated weights for policy 0, policy_version 63960 (0.0009) -[2023-10-15 17:20:45,852][52866] Updated weights for policy 1, policy_version 64180 (0.0007) -[2023-10-15 17:20:46,212][52866] Updated weights for policy 1, policy_version 64190 (0.0010) -[2023-10-15 17:20:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 131235840. Throughput: 0: 1785.5, 1: 1798.4. Samples: 32812846. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:20:48,442][51532] Avg episode reward: [(0, '55.950'), (1, '63.640')] -[2023-10-15 17:20:49,751][52833] Updated weights for policy 0, policy_version 63970 (0.0007) -[2023-10-15 17:20:50,118][52833] Updated weights for policy 0, policy_version 63980 (0.0008) -[2023-10-15 17:20:50,122][52866] Updated weights for policy 1, policy_version 64200 (0.0009) -[2023-10-15 17:20:50,488][52833] Updated weights for policy 0, policy_version 63990 (0.0007) -[2023-10-15 17:20:50,495][52866] Updated weights for policy 1, policy_version 64210 (0.0008) -[2023-10-15 17:20:50,853][52866] Updated weights for policy 1, policy_version 64220 (0.0007) -[2023-10-15 17:20:50,859][52833] Updated weights for policy 0, policy_version 64000 (0.0007) -[2023-10-15 17:20:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 131301376. Throughput: 0: 1776.0, 1: 1784.0. Samples: 32834194. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:20:53,441][51532] Avg episode reward: [(0, '58.380'), (1, '63.520')] -[2023-10-15 17:20:54,586][52866] Updated weights for policy 1, policy_version 64230 (0.0009) -[2023-10-15 17:20:54,648][52833] Updated weights for policy 0, policy_version 64010 (0.0008) -[2023-10-15 17:20:54,948][52866] Updated weights for policy 1, policy_version 64240 (0.0008) -[2023-10-15 17:20:55,009][52833] Updated weights for policy 0, policy_version 64020 (0.0009) -[2023-10-15 17:20:55,318][52866] Updated weights for policy 1, policy_version 64250 (0.0007) -[2023-10-15 17:20:55,381][52833] Updated weights for policy 0, policy_version 64030 (0.0008) -[2023-10-15 17:20:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 131366912. Throughput: 0: 1780.4, 1: 1786.0. Samples: 32856966. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 17:20:58,442][51532] Avg episode reward: [(0, '54.600'), (1, '63.020')] -[2023-10-15 17:20:58,911][52866] Updated weights for policy 1, policy_version 64260 (0.0008) -[2023-10-15 17:20:59,185][52833] Updated weights for policy 0, policy_version 64040 (0.0009) -[2023-10-15 17:20:59,273][52866] Updated weights for policy 1, policy_version 64270 (0.0008) -[2023-10-15 17:20:59,552][52833] Updated weights for policy 0, policy_version 64050 (0.0009) -[2023-10-15 17:20:59,644][52866] Updated weights for policy 1, policy_version 64280 (0.0007) -[2023-10-15 17:20:59,933][52833] Updated weights for policy 0, policy_version 64060 (0.0008) -[2023-10-15 17:21:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 131432448. Throughput: 0: 1781.8, 1: 1787.6. Samples: 32866776. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 17:21:03,441][51532] Avg episode reward: [(0, '52.690'), (1, '63.700')] -[2023-10-15 17:21:03,476][52866] Updated weights for policy 1, policy_version 64290 (0.0009) -[2023-10-15 17:21:03,598][52833] Updated weights for policy 0, policy_version 64070 (0.0007) -[2023-10-15 17:21:03,847][52866] Updated weights for policy 1, policy_version 64300 (0.0008) -[2023-10-15 17:21:03,967][52833] Updated weights for policy 0, policy_version 64080 (0.0011) -[2023-10-15 17:21:04,210][52866] Updated weights for policy 1, policy_version 64310 (0.0008) -[2023-10-15 17:21:04,342][52833] Updated weights for policy 0, policy_version 64090 (0.0007) -[2023-10-15 17:21:04,577][52866] Updated weights for policy 1, policy_version 64320 (0.0008) -[2023-10-15 17:21:08,012][52833] Updated weights for policy 0, policy_version 64100 (0.0007) -[2023-10-15 17:21:08,332][52866] Updated weights for policy 1, policy_version 64330 (0.0009) -[2023-10-15 17:21:08,374][52833] Updated weights for policy 0, policy_version 64110 (0.0007) -[2023-10-15 17:21:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 131497984. Throughput: 0: 1786.9, 1: 1782.1. Samples: 32889100. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 17:21:08,442][51532] Avg episode reward: [(0, '53.020'), (1, '63.130')] -[2023-10-15 17:21:08,701][52866] Updated weights for policy 1, policy_version 64340 (0.0008) -[2023-10-15 17:21:08,742][52833] Updated weights for policy 0, policy_version 64120 (0.0007) -[2023-10-15 17:21:09,059][52866] Updated weights for policy 1, policy_version 64350 (0.0008) -[2023-10-15 17:21:12,590][52833] Updated weights for policy 0, policy_version 64130 (0.0008) -[2023-10-15 17:21:12,820][52866] Updated weights for policy 1, policy_version 64360 (0.0008) -[2023-10-15 17:21:12,957][52833] Updated weights for policy 0, policy_version 64140 (0.0008) -[2023-10-15 17:21:13,185][52866] Updated weights for policy 1, policy_version 64370 (0.0009) -[2023-10-15 17:21:13,331][52833] Updated weights for policy 0, policy_version 64150 (0.0009) -[2023-10-15 17:21:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 131563520. Throughput: 0: 1810.9, 1: 1804.0. Samples: 32910554. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 17:21:13,442][51532] Avg episode reward: [(0, '55.770'), (1, '61.140')] -[2023-10-15 17:21:13,551][52866] Updated weights for policy 1, policy_version 64380 (0.0007) -[2023-10-15 17:21:13,698][52833] Updated weights for policy 0, policy_version 64160 (0.0008) -[2023-10-15 17:21:17,211][52866] Updated weights for policy 1, policy_version 64390 (0.0008) -[2023-10-15 17:21:17,482][52833] Updated weights for policy 0, policy_version 64170 (0.0007) -[2023-10-15 17:21:17,576][52866] Updated weights for policy 1, policy_version 64400 (0.0007) -[2023-10-15 17:21:17,849][52833] Updated weights for policy 0, policy_version 64180 (0.0008) -[2023-10-15 17:21:17,930][52866] Updated weights for policy 1, policy_version 64410 (0.0007) -[2023-10-15 17:21:18,212][52833] Updated weights for policy 0, policy_version 64190 (0.0008) -[2023-10-15 17:21:18,441][51532] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 131694592. Throughput: 0: 1789.1, 1: 1785.5. Samples: 32921232. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 17:21:18,442][51532] Avg episode reward: [(0, '56.790'), (1, '59.900')] -[2023-10-15 17:21:21,640][52866] Updated weights for policy 1, policy_version 64420 (0.0009) -[2023-10-15 17:21:21,856][52833] Updated weights for policy 0, policy_version 64200 (0.0009) -[2023-10-15 17:21:22,011][52866] Updated weights for policy 1, policy_version 64430 (0.0007) -[2023-10-15 17:21:22,233][52833] Updated weights for policy 0, policy_version 64210 (0.0009) -[2023-10-15 17:21:22,367][52866] Updated weights for policy 1, policy_version 64440 (0.0007) -[2023-10-15 17:21:22,601][52833] Updated weights for policy 0, policy_version 64220 (0.0009) -[2023-10-15 17:21:23,441][51532] Fps is (10 sec: 19660.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 131760128. Throughput: 0: 1805.7, 1: 1801.6. Samples: 32942780. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 17:21:23,442][51532] Avg episode reward: [(0, '57.740'), (1, '60.190')] -[2023-10-15 17:21:26,055][52866] Updated weights for policy 1, policy_version 64450 (0.0007) -[2023-10-15 17:21:26,303][52833] Updated weights for policy 0, policy_version 64230 (0.0008) -[2023-10-15 17:21:26,418][52866] Updated weights for policy 1, policy_version 64460 (0.0010) -[2023-10-15 17:21:26,668][52833] Updated weights for policy 0, policy_version 64240 (0.0008) -[2023-10-15 17:21:26,790][52866] Updated weights for policy 1, policy_version 64470 (0.0007) -[2023-10-15 17:21:27,042][52833] Updated weights for policy 0, policy_version 64250 (0.0008) -[2023-10-15 17:21:27,155][52866] Updated weights for policy 1, policy_version 64480 (0.0007) -[2023-10-15 17:21:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 131825664. Throughput: 0: 1785.0, 1: 1797.3. Samples: 32963428. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 17:21:28,441][51532] Avg episode reward: [(0, '56.120'), (1, '62.330')] -[2023-10-15 17:21:30,836][52866] Updated weights for policy 1, policy_version 64490 (0.0008) -[2023-10-15 17:21:30,886][52833] Updated weights for policy 0, policy_version 64260 (0.0008) -[2023-10-15 17:21:31,207][52866] Updated weights for policy 1, policy_version 64500 (0.0008) -[2023-10-15 17:21:31,248][52833] Updated weights for policy 0, policy_version 64270 (0.0009) -[2023-10-15 17:21:31,562][52866] Updated weights for policy 1, policy_version 64510 (0.0007) -[2023-10-15 17:21:31,612][52833] Updated weights for policy 0, policy_version 64280 (0.0007) -[2023-10-15 17:21:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 131891200. Throughput: 0: 1806.7, 1: 1805.6. Samples: 32975400. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-15 17:21:33,442][51532] Avg episode reward: [(0, '55.880'), (1, '62.920')] -[2023-10-15 17:21:35,236][52833] Updated weights for policy 0, policy_version 64290 (0.0009) -[2023-10-15 17:21:35,509][52866] Updated weights for policy 1, policy_version 64520 (0.0007) -[2023-10-15 17:21:35,597][52833] Updated weights for policy 0, policy_version 64300 (0.0007) -[2023-10-15 17:21:35,874][52866] Updated weights for policy 1, policy_version 64530 (0.0008) -[2023-10-15 17:21:35,973][52833] Updated weights for policy 0, policy_version 64310 (0.0007) -[2023-10-15 17:21:36,234][52866] Updated weights for policy 1, policy_version 64540 (0.0009) -[2023-10-15 17:21:36,340][52833] Updated weights for policy 0, policy_version 64320 (0.0008) -[2023-10-15 17:21:38,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 131956736. Throughput: 0: 1790.4, 1: 1795.1. Samples: 32995544. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 17:21:38,442][51532] Avg episode reward: [(0, '57.310'), (1, '61.910')] -[2023-10-15 17:21:39,995][52866] Updated weights for policy 1, policy_version 64550 (0.0008) -[2023-10-15 17:21:40,101][52833] Updated weights for policy 0, policy_version 64330 (0.0007) -[2023-10-15 17:21:40,371][52866] Updated weights for policy 1, policy_version 64560 (0.0009) -[2023-10-15 17:21:40,464][52833] Updated weights for policy 0, policy_version 64340 (0.0007) -[2023-10-15 17:21:40,743][52866] Updated weights for policy 1, policy_version 64570 (0.0007) -[2023-10-15 17:21:40,829][52833] Updated weights for policy 0, policy_version 64350 (0.0007) -[2023-10-15 17:21:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 132022272. Throughput: 0: 1790.0, 1: 1786.2. Samples: 33017896. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 17:21:43,442][51532] Avg episode reward: [(0, '55.000'), (1, '58.950')] -[2023-10-15 17:21:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000064352_65896448.pth... -[2023-10-15 17:21:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000064576_66125824.pth... -[2023-10-15 17:21:43,488][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000062688_64192512.pth -[2023-10-15 17:21:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000062912_64421888.pth -[2023-10-15 17:21:44,576][52833] Updated weights for policy 0, policy_version 64360 (0.0008) -[2023-10-15 17:21:44,597][52866] Updated weights for policy 1, policy_version 64580 (0.0007) -[2023-10-15 17:21:44,933][52833] Updated weights for policy 0, policy_version 64370 (0.0009) -[2023-10-15 17:21:44,960][52866] Updated weights for policy 1, policy_version 64590 (0.0009) -[2023-10-15 17:21:45,306][52833] Updated weights for policy 0, policy_version 64380 (0.0007) -[2023-10-15 17:21:45,328][52866] Updated weights for policy 1, policy_version 64600 (0.0007) -[2023-10-15 17:21:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 132087808. Throughput: 0: 1790.4, 1: 1778.4. Samples: 33027370. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 17:21:48,441][51532] Avg episode reward: [(0, '57.930'), (1, '58.140')] -[2023-10-15 17:21:49,055][52833] Updated weights for policy 0, policy_version 64390 (0.0009) -[2023-10-15 17:21:49,226][52866] Updated weights for policy 1, policy_version 64610 (0.0009) -[2023-10-15 17:21:49,411][52833] Updated weights for policy 0, policy_version 64400 (0.0008) -[2023-10-15 17:21:49,591][52866] Updated weights for policy 1, policy_version 64620 (0.0008) -[2023-10-15 17:21:49,787][52833] Updated weights for policy 0, policy_version 64410 (0.0008) -[2023-10-15 17:21:49,960][52866] Updated weights for policy 1, policy_version 64630 (0.0009) -[2023-10-15 17:21:50,338][52866] Updated weights for policy 1, policy_version 64640 (0.0010) -[2023-10-15 17:21:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 132153344. Throughput: 0: 1785.3, 1: 1779.8. Samples: 33049530. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 17:21:53,442][51532] Avg episode reward: [(0, '59.220'), (1, '57.790')] -[2023-10-15 17:21:53,490][52833] Updated weights for policy 0, policy_version 64420 (0.0009) -[2023-10-15 17:21:53,863][52833] Updated weights for policy 0, policy_version 64430 (0.0007) -[2023-10-15 17:21:54,126][52866] Updated weights for policy 1, policy_version 64650 (0.0007) -[2023-10-15 17:21:54,226][52833] Updated weights for policy 0, policy_version 64440 (0.0008) -[2023-10-15 17:21:54,497][52866] Updated weights for policy 1, policy_version 64660 (0.0008) -[2023-10-15 17:21:54,864][52866] Updated weights for policy 1, policy_version 64670 (0.0008) -[2023-10-15 17:21:58,031][52833] Updated weights for policy 0, policy_version 64450 (0.0008) -[2023-10-15 17:21:58,403][52833] Updated weights for policy 0, policy_version 64460 (0.0008) -[2023-10-15 17:21:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 132218880. Throughput: 0: 1793.6, 1: 1795.9. Samples: 33072078. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 17:21:58,441][51532] Avg episode reward: [(0, '59.700'), (1, '57.190')] -[2023-10-15 17:21:58,642][52866] Updated weights for policy 1, policy_version 64680 (0.0009) -[2023-10-15 17:21:58,763][52833] Updated weights for policy 0, policy_version 64470 (0.0008) -[2023-10-15 17:21:59,006][52866] Updated weights for policy 1, policy_version 64690 (0.0009) -[2023-10-15 17:21:59,134][52833] Updated weights for policy 0, policy_version 64480 (0.0007) -[2023-10-15 17:21:59,388][52866] Updated weights for policy 1, policy_version 64700 (0.0008) -[2023-10-15 17:22:03,073][52833] Updated weights for policy 0, policy_version 64490 (0.0008) -[2023-10-15 17:22:03,080][52866] Updated weights for policy 1, policy_version 64710 (0.0008) -[2023-10-15 17:22:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 132284416. Throughput: 0: 1787.0, 1: 1775.9. Samples: 33081564. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 17:22:03,442][51532] Avg episode reward: [(0, '59.100'), (1, '56.310')] -[2023-10-15 17:22:03,442][52866] Updated weights for policy 1, policy_version 64720 (0.0007) -[2023-10-15 17:22:03,443][52833] Updated weights for policy 0, policy_version 64500 (0.0008) -[2023-10-15 17:22:03,809][52866] Updated weights for policy 1, policy_version 64730 (0.0007) -[2023-10-15 17:22:03,811][52833] Updated weights for policy 0, policy_version 64510 (0.0008) -[2023-10-15 17:22:07,550][52833] Updated weights for policy 0, policy_version 64520 (0.0007) -[2023-10-15 17:22:07,595][52866] Updated weights for policy 1, policy_version 64740 (0.0007) -[2023-10-15 17:22:07,920][52833] Updated weights for policy 0, policy_version 64530 (0.0008) -[2023-10-15 17:22:07,961][52866] Updated weights for policy 1, policy_version 64750 (0.0007) -[2023-10-15 17:22:08,281][52833] Updated weights for policy 0, policy_version 64540 (0.0010) -[2023-10-15 17:22:08,330][52866] Updated weights for policy 1, policy_version 64760 (0.0007) -[2023-10-15 17:22:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 132382720. Throughput: 0: 1794.8, 1: 1790.5. Samples: 33104120. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 17:22:08,441][51532] Avg episode reward: [(0, '59.290'), (1, '55.560')] -[2023-10-15 17:22:11,956][52833] Updated weights for policy 0, policy_version 64550 (0.0009) -[2023-10-15 17:22:12,050][52866] Updated weights for policy 1, policy_version 64770 (0.0010) -[2023-10-15 17:22:12,323][52833] Updated weights for policy 0, policy_version 64560 (0.0007) -[2023-10-15 17:22:12,420][52866] Updated weights for policy 1, policy_version 64780 (0.0007) -[2023-10-15 17:22:12,689][52833] Updated weights for policy 0, policy_version 64570 (0.0007) -[2023-10-15 17:22:12,781][52866] Updated weights for policy 1, policy_version 64790 (0.0007) -[2023-10-15 17:22:13,145][52866] Updated weights for policy 1, policy_version 64800 (0.0009) -[2023-10-15 17:22:13,441][51532] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 132481024. Throughput: 0: 1791.9, 1: 1774.6. Samples: 33123920. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-15 17:22:13,442][51532] Avg episode reward: [(0, '56.080'), (1, '57.060')] -[2023-10-15 17:22:16,399][52833] Updated weights for policy 0, policy_version 64580 (0.0007) -[2023-10-15 17:22:16,770][52866] Updated weights for policy 1, policy_version 64810 (0.0008) -[2023-10-15 17:22:16,772][52833] Updated weights for policy 0, policy_version 64590 (0.0009) -[2023-10-15 17:22:17,132][52866] Updated weights for policy 1, policy_version 64820 (0.0008) -[2023-10-15 17:22:17,140][52833] Updated weights for policy 0, policy_version 64600 (0.0009) -[2023-10-15 17:22:17,505][52866] Updated weights for policy 1, policy_version 64830 (0.0008) -[2023-10-15 17:22:18,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132546560. Throughput: 0: 1792.1, 1: 1782.5. Samples: 33136258. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:18,441][51532] Avg episode reward: [(0, '58.960'), (1, '58.770')] -[2023-10-15 17:22:20,888][52833] Updated weights for policy 0, policy_version 64610 (0.0008) -[2023-10-15 17:22:21,193][52866] Updated weights for policy 1, policy_version 64840 (0.0007) -[2023-10-15 17:22:21,248][52833] Updated weights for policy 0, policy_version 64620 (0.0008) -[2023-10-15 17:22:21,561][52866] Updated weights for policy 1, policy_version 64850 (0.0007) -[2023-10-15 17:22:21,613][52833] Updated weights for policy 0, policy_version 64630 (0.0007) -[2023-10-15 17:22:21,918][52866] Updated weights for policy 1, policy_version 64860 (0.0008) -[2023-10-15 17:22:21,978][52833] Updated weights for policy 0, policy_version 64640 (0.0007) -[2023-10-15 17:22:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132612096. Throughput: 0: 1788.1, 1: 1781.1. Samples: 33156160. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:23,442][51532] Avg episode reward: [(0, '56.410'), (1, '56.950')] -[2023-10-15 17:22:25,654][52833] Updated weights for policy 0, policy_version 64650 (0.0008) -[2023-10-15 17:22:25,656][52866] Updated weights for policy 1, policy_version 64870 (0.0008) -[2023-10-15 17:22:26,018][52833] Updated weights for policy 0, policy_version 64660 (0.0009) -[2023-10-15 17:22:26,029][52866] Updated weights for policy 1, policy_version 64880 (0.0010) -[2023-10-15 17:22:26,392][52866] Updated weights for policy 1, policy_version 64890 (0.0009) -[2023-10-15 17:22:26,393][52833] Updated weights for policy 0, policy_version 64670 (0.0009) -[2023-10-15 17:22:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 132677632. Throughput: 0: 1781.1, 1: 1783.1. Samples: 33178284. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:28,442][51532] Avg episode reward: [(0, '57.970'), (1, '55.870')] -[2023-10-15 17:22:30,276][52866] Updated weights for policy 1, policy_version 64900 (0.0009) -[2023-10-15 17:22:30,389][52833] Updated weights for policy 0, policy_version 64680 (0.0007) -[2023-10-15 17:22:30,643][52866] Updated weights for policy 1, policy_version 64910 (0.0009) -[2023-10-15 17:22:30,768][52833] Updated weights for policy 0, policy_version 64690 (0.0009) -[2023-10-15 17:22:31,013][52866] Updated weights for policy 1, policy_version 64920 (0.0009) -[2023-10-15 17:22:31,129][52833] Updated weights for policy 0, policy_version 64700 (0.0008) -[2023-10-15 17:22:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132743168. Throughput: 0: 1791.0, 1: 1795.8. Samples: 33188776. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:33,441][51532] Avg episode reward: [(0, '58.900'), (1, '54.680')] -[2023-10-15 17:22:34,682][52866] Updated weights for policy 1, policy_version 64930 (0.0010) -[2023-10-15 17:22:34,869][52833] Updated weights for policy 0, policy_version 64710 (0.0008) -[2023-10-15 17:22:35,055][52866] Updated weights for policy 1, policy_version 64940 (0.0009) -[2023-10-15 17:22:35,240][52833] Updated weights for policy 0, policy_version 64720 (0.0009) -[2023-10-15 17:22:35,416][52866] Updated weights for policy 1, policy_version 64950 (0.0010) -[2023-10-15 17:22:35,609][52833] Updated weights for policy 0, policy_version 64730 (0.0008) -[2023-10-15 17:22:35,775][52866] Updated weights for policy 1, policy_version 64960 (0.0008) -[2023-10-15 17:22:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 132808704. Throughput: 0: 1784.0, 1: 1790.9. Samples: 33210402. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:38,442][51532] Avg episode reward: [(0, '60.640'), (1, '55.350')] -[2023-10-15 17:22:39,375][52833] Updated weights for policy 0, policy_version 64740 (0.0008) -[2023-10-15 17:22:39,479][52866] Updated weights for policy 1, policy_version 64970 (0.0007) -[2023-10-15 17:22:39,741][52833] Updated weights for policy 0, policy_version 64750 (0.0007) -[2023-10-15 17:22:39,845][52866] Updated weights for policy 1, policy_version 64980 (0.0009) -[2023-10-15 17:22:40,118][52833] Updated weights for policy 0, policy_version 64760 (0.0007) -[2023-10-15 17:22:40,219][52866] Updated weights for policy 1, policy_version 64990 (0.0008) -[2023-10-15 17:22:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 132874240. Throughput: 0: 1786.3, 1: 1785.1. Samples: 33232790. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:43,442][51532] Avg episode reward: [(0, '63.440'), (1, '56.170')] -[2023-10-15 17:22:44,048][52833] Updated weights for policy 0, policy_version 64770 (0.0009) -[2023-10-15 17:22:44,081][52866] Updated weights for policy 1, policy_version 65000 (0.0008) -[2023-10-15 17:22:44,418][52833] Updated weights for policy 0, policy_version 64780 (0.0008) -[2023-10-15 17:22:44,441][52866] Updated weights for policy 1, policy_version 65010 (0.0009) -[2023-10-15 17:22:44,783][52833] Updated weights for policy 0, policy_version 64790 (0.0008) -[2023-10-15 17:22:44,805][52866] Updated weights for policy 1, policy_version 65020 (0.0007) -[2023-10-15 17:22:45,152][52833] Updated weights for policy 0, policy_version 64800 (0.0009) -[2023-10-15 17:22:48,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 132939776. Throughput: 0: 1784.1, 1: 1789.5. Samples: 33242372. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:48,442][51532] Avg episode reward: [(0, '62.880'), (1, '56.100')] -[2023-10-15 17:22:48,688][52866] Updated weights for policy 1, policy_version 65030 (0.0008) -[2023-10-15 17:22:48,898][52833] Updated weights for policy 0, policy_version 64810 (0.0009) -[2023-10-15 17:22:49,059][52866] Updated weights for policy 1, policy_version 65040 (0.0007) -[2023-10-15 17:22:49,268][52833] Updated weights for policy 0, policy_version 64820 (0.0008) -[2023-10-15 17:22:49,424][52866] Updated weights for policy 1, policy_version 65050 (0.0007) -[2023-10-15 17:22:49,637][52833] Updated weights for policy 0, policy_version 64830 (0.0009) -[2023-10-15 17:22:53,093][52866] Updated weights for policy 1, policy_version 65060 (0.0007) -[2023-10-15 17:22:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 133005312. Throughput: 0: 1782.0, 1: 1787.9. Samples: 33264764. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:53,442][51532] Avg episode reward: [(0, '64.620'), (1, '55.900')] -[2023-10-15 17:22:53,451][52866] Updated weights for policy 1, policy_version 65070 (0.0007) -[2023-10-15 17:22:53,602][52833] Updated weights for policy 0, policy_version 64840 (0.0009) -[2023-10-15 17:22:53,821][52866] Updated weights for policy 1, policy_version 65080 (0.0007) -[2023-10-15 17:22:53,966][52833] Updated weights for policy 0, policy_version 64850 (0.0008) -[2023-10-15 17:22:54,340][52833] Updated weights for policy 0, policy_version 64860 (0.0007) -[2023-10-15 17:22:57,657][52866] Updated weights for policy 1, policy_version 65090 (0.0009) -[2023-10-15 17:22:58,012][52866] Updated weights for policy 1, policy_version 65100 (0.0008) -[2023-10-15 17:22:58,149][52833] Updated weights for policy 0, policy_version 64870 (0.0008) -[2023-10-15 17:22:58,375][52866] Updated weights for policy 1, policy_version 65110 (0.0008) -[2023-10-15 17:22:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 133070848. Throughput: 0: 1811.3, 1: 1804.5. Samples: 33286632. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-15 17:22:58,442][51532] Avg episode reward: [(0, '62.900'), (1, '58.770')] -[2023-10-15 17:22:58,518][52833] Updated weights for policy 0, policy_version 64880 (0.0008) -[2023-10-15 17:22:58,735][52866] Updated weights for policy 1, policy_version 65120 (0.0008) -[2023-10-15 17:22:58,880][52833] Updated weights for policy 0, policy_version 64890 (0.0008) -[2023-10-15 17:23:02,486][52866] Updated weights for policy 1, policy_version 65130 (0.0008) -[2023-10-15 17:23:02,523][52833] Updated weights for policy 0, policy_version 64900 (0.0008) -[2023-10-15 17:23:02,862][52866] Updated weights for policy 1, policy_version 65140 (0.0007) -[2023-10-15 17:23:02,889][52833] Updated weights for policy 0, policy_version 64910 (0.0008) -[2023-10-15 17:23:03,232][52866] Updated weights for policy 1, policy_version 65150 (0.0007) -[2023-10-15 17:23:03,251][52833] Updated weights for policy 0, policy_version 64920 (0.0007) -[2023-10-15 17:23:03,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 133169152. Throughput: 0: 1779.1, 1: 1792.9. Samples: 33296996. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) -[2023-10-15 17:23:03,441][51532] Avg episode reward: [(0, '65.440'), (1, '62.940')] -[2023-10-15 17:23:06,926][52833] Updated weights for policy 0, policy_version 64930 (0.0007) -[2023-10-15 17:23:07,002][52866] Updated weights for policy 1, policy_version 65160 (0.0009) -[2023-10-15 17:23:07,299][52833] Updated weights for policy 0, policy_version 64940 (0.0009) -[2023-10-15 17:23:07,364][52866] Updated weights for policy 1, policy_version 65170 (0.0008) -[2023-10-15 17:23:07,668][52833] Updated weights for policy 0, policy_version 64950 (0.0007) -[2023-10-15 17:23:07,728][52866] Updated weights for policy 1, policy_version 65180 (0.0010) -[2023-10-15 17:23:08,040][52833] Updated weights for policy 0, policy_version 64960 (0.0009) -[2023-10-15 17:23:08,441][51532] Fps is (10 sec: 19661.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 133267456. Throughput: 0: 1811.3, 1: 1810.1. Samples: 33319122. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) -[2023-10-15 17:23:08,441][51532] Avg episode reward: [(0, '65.340'), (1, '64.750')] -[2023-10-15 17:23:11,458][52866] Updated weights for policy 1, policy_version 65190 (0.0009) -[2023-10-15 17:23:11,589][52833] Updated weights for policy 0, policy_version 64970 (0.0010) -[2023-10-15 17:23:11,835][52866] Updated weights for policy 1, policy_version 65200 (0.0008) -[2023-10-15 17:23:11,950][52833] Updated weights for policy 0, policy_version 64980 (0.0009) -[2023-10-15 17:23:12,196][52866] Updated weights for policy 1, policy_version 65210 (0.0009) -[2023-10-15 17:23:12,334][52833] Updated weights for policy 0, policy_version 64990 (0.0008) -[2023-10-15 17:23:13,441][51532] Fps is (10 sec: 16383.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 133332992. Throughput: 0: 1782.6, 1: 1786.5. Samples: 33338894. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) -[2023-10-15 17:23:13,443][51532] Avg episode reward: [(0, '67.770'), (1, '66.010')] -[2023-10-15 17:23:13,455][52410] Saving new best policy, reward=67.770! -[2023-10-15 17:23:16,001][52866] Updated weights for policy 1, policy_version 65220 (0.0009) -[2023-10-15 17:23:16,192][52833] Updated weights for policy 0, policy_version 65000 (0.0009) -[2023-10-15 17:23:16,363][52866] Updated weights for policy 1, policy_version 65230 (0.0008) -[2023-10-15 17:23:16,546][52833] Updated weights for policy 0, policy_version 65010 (0.0008) -[2023-10-15 17:23:16,724][52866] Updated weights for policy 1, policy_version 65240 (0.0009) -[2023-10-15 17:23:16,922][52833] Updated weights for policy 0, policy_version 65020 (0.0008) -[2023-10-15 17:23:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.2). Total num frames: 133398528. Throughput: 0: 1805.6, 1: 1809.0. Samples: 33351434. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) -[2023-10-15 17:23:18,441][51532] Avg episode reward: [(0, '68.000'), (1, '67.790')] -[2023-10-15 17:23:18,442][52410] Saving new best policy, reward=68.000! -[2023-10-15 17:23:20,441][52866] Updated weights for policy 1, policy_version 65250 (0.0010) -[2023-10-15 17:23:20,713][52833] Updated weights for policy 0, policy_version 65030 (0.0010) -[2023-10-15 17:23:20,808][52866] Updated weights for policy 1, policy_version 65260 (0.0007) -[2023-10-15 17:23:21,084][52833] Updated weights for policy 0, policy_version 65040 (0.0009) -[2023-10-15 17:23:21,179][52866] Updated weights for policy 1, policy_version 65270 (0.0007) -[2023-10-15 17:23:21,444][52833] Updated weights for policy 0, policy_version 65050 (0.0007) -[2023-10-15 17:23:21,532][52866] Updated weights for policy 1, policy_version 65280 (0.0008) -[2023-10-15 17:23:23,441][51532] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 133464064. Throughput: 0: 1782.2, 1: 1790.2. Samples: 33371160. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) -[2023-10-15 17:23:23,441][51532] Avg episode reward: [(0, '66.620'), (1, '67.280')] -[2023-10-15 17:23:25,357][52866] Updated weights for policy 1, policy_version 65290 (0.0009) -[2023-10-15 17:23:25,384][52833] Updated weights for policy 0, policy_version 65060 (0.0009) -[2023-10-15 17:23:25,715][52866] Updated weights for policy 1, policy_version 65300 (0.0008) -[2023-10-15 17:23:25,756][52833] Updated weights for policy 0, policy_version 65070 (0.0007) -[2023-10-15 17:23:26,077][52866] Updated weights for policy 1, policy_version 65310 (0.0009) -[2023-10-15 17:23:26,128][52833] Updated weights for policy 0, policy_version 65080 (0.0007) -[2023-10-15 17:23:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 133529600. Throughput: 0: 1776.4, 1: 1790.3. Samples: 33393290. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) -[2023-10-15 17:23:28,442][51532] Avg episode reward: [(0, '62.680'), (1, '70.660')] -[2023-10-15 17:23:28,450][52518] Saving new best policy, reward=70.660! -[2023-10-15 17:23:30,018][52866] Updated weights for policy 1, policy_version 65320 (0.0008) -[2023-10-15 17:23:30,037][52833] Updated weights for policy 0, policy_version 65090 (0.0007) -[2023-10-15 17:23:30,395][52866] Updated weights for policy 1, policy_version 65330 (0.0007) -[2023-10-15 17:23:30,405][52833] Updated weights for policy 0, policy_version 65100 (0.0007) -[2023-10-15 17:23:30,757][52866] Updated weights for policy 1, policy_version 65340 (0.0007) -[2023-10-15 17:23:30,766][52833] Updated weights for policy 0, policy_version 65110 (0.0008) -[2023-10-15 17:23:31,133][52833] Updated weights for policy 0, policy_version 65120 (0.0009) -[2023-10-15 17:23:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 133595136. Throughput: 0: 1787.1, 1: 1788.4. Samples: 33403268. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) -[2023-10-15 17:23:33,441][51532] Avg episode reward: [(0, '63.720'), (1, '69.360')] -[2023-10-15 17:23:34,376][52866] Updated weights for policy 1, policy_version 65350 (0.0008) -[2023-10-15 17:23:34,744][52866] Updated weights for policy 1, policy_version 65360 (0.0008) -[2023-10-15 17:23:34,818][52833] Updated weights for policy 0, policy_version 65130 (0.0008) -[2023-10-15 17:23:35,113][52866] Updated weights for policy 1, policy_version 65370 (0.0008) -[2023-10-15 17:23:35,194][52833] Updated weights for policy 0, policy_version 65140 (0.0009) -[2023-10-15 17:23:35,562][52833] Updated weights for policy 0, policy_version 65150 (0.0009) -[2023-10-15 17:23:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 133660672. Throughput: 0: 1778.7, 1: 1790.3. Samples: 33425366. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) -[2023-10-15 17:23:38,442][51532] Avg episode reward: [(0, '61.790'), (1, '69.370')] -[2023-10-15 17:23:38,963][52866] Updated weights for policy 1, policy_version 65380 (0.0007) -[2023-10-15 17:23:39,218][52833] Updated weights for policy 0, policy_version 65160 (0.0007) -[2023-10-15 17:23:39,321][52866] Updated weights for policy 1, policy_version 65390 (0.0009) -[2023-10-15 17:23:39,582][52833] Updated weights for policy 0, policy_version 65170 (0.0009) -[2023-10-15 17:23:39,679][52866] Updated weights for policy 1, policy_version 65400 (0.0008) -[2023-10-15 17:23:39,948][52833] Updated weights for policy 0, policy_version 65180 (0.0008) -[2023-10-15 17:23:43,422][52866] Updated weights for policy 1, policy_version 65410 (0.0008) -[2023-10-15 17:23:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 133726208. Throughput: 0: 1781.7, 1: 1797.7. Samples: 33447702. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 17:23:43,441][51532] Avg episode reward: [(0, '62.690'), (1, '72.030')] -[2023-10-15 17:23:43,722][52833] Updated weights for policy 0, policy_version 65190 (0.0009) -[2023-10-15 17:23:43,785][52866] Updated weights for policy 1, policy_version 65420 (0.0008) -[2023-10-15 17:23:44,090][52833] Updated weights for policy 0, policy_version 65200 (0.0008) -[2023-10-15 17:23:44,141][52866] Updated weights for policy 1, policy_version 65430 (0.0008) -[2023-10-15 17:23:44,462][52833] Updated weights for policy 0, policy_version 65210 (0.0007) -[2023-10-15 17:23:44,507][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000065440_67010560.pth... -[2023-10-15 17:23:44,512][52866] Updated weights for policy 1, policy_version 65440 (0.0007) -[2023-10-15 17:23:44,535][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000063744_65273856.pth -[2023-10-15 17:23:44,538][52518] Saving new best policy, reward=72.030! -[2023-10-15 17:23:44,676][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000065216_66781184.pth... -[2023-10-15 17:23:44,711][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000063520_65044480.pth -[2023-10-15 17:23:48,188][52866] Updated weights for policy 1, policy_version 65450 (0.0008) -[2023-10-15 17:23:48,366][52833] Updated weights for policy 0, policy_version 65220 (0.0007) -[2023-10-15 17:23:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 133791744. Throughput: 0: 1787.8, 1: 1782.0. Samples: 33457636. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 17:23:48,442][51532] Avg episode reward: [(0, '62.130'), (1, '73.450')] -[2023-10-15 17:23:48,552][52866] Updated weights for policy 1, policy_version 65460 (0.0008) -[2023-10-15 17:23:48,725][52833] Updated weights for policy 0, policy_version 65230 (0.0009) -[2023-10-15 17:23:48,920][52866] Updated weights for policy 1, policy_version 65470 (0.0009) -[2023-10-15 17:23:48,992][52518] Saving new best policy, reward=73.450! -[2023-10-15 17:23:49,093][52833] Updated weights for policy 0, policy_version 65240 (0.0009) -[2023-10-15 17:23:52,751][52866] Updated weights for policy 1, policy_version 65480 (0.0010) -[2023-10-15 17:23:52,875][52833] Updated weights for policy 0, policy_version 65250 (0.0007) -[2023-10-15 17:23:53,119][52866] Updated weights for policy 1, policy_version 65490 (0.0007) -[2023-10-15 17:23:53,244][52833] Updated weights for policy 0, policy_version 65260 (0.0007) -[2023-10-15 17:23:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 133857280. Throughput: 0: 1780.0, 1: 1792.4. Samples: 33479876. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 17:23:53,441][51532] Avg episode reward: [(0, '61.180'), (1, '73.730')] -[2023-10-15 17:23:53,498][52866] Updated weights for policy 1, policy_version 65500 (0.0008) -[2023-10-15 17:23:53,612][52833] Updated weights for policy 0, policy_version 65270 (0.0007) -[2023-10-15 17:23:53,634][52518] Saving new best policy, reward=73.730! -[2023-10-15 17:23:53,976][52833] Updated weights for policy 0, policy_version 65280 (0.0009) -[2023-10-15 17:23:57,411][52866] Updated weights for policy 1, policy_version 65510 (0.0009) -[2023-10-15 17:23:57,775][52866] Updated weights for policy 1, policy_version 65520 (0.0007) -[2023-10-15 17:23:57,785][52833] Updated weights for policy 0, policy_version 65290 (0.0007) -[2023-10-15 17:23:58,141][52866] Updated weights for policy 1, policy_version 65530 (0.0007) -[2023-10-15 17:23:58,151][52833] Updated weights for policy 0, policy_version 65300 (0.0008) -[2023-10-15 17:23:58,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 133955584. Throughput: 0: 1801.7, 1: 1792.9. Samples: 33500648. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 17:23:58,442][51532] Avg episode reward: [(0, '60.390'), (1, '73.700')] -[2023-10-15 17:23:58,522][52833] Updated weights for policy 0, policy_version 65310 (0.0010) -[2023-10-15 17:24:01,805][52866] Updated weights for policy 1, policy_version 65540 (0.0007) -[2023-10-15 17:24:02,178][52866] Updated weights for policy 1, policy_version 65550 (0.0008) -[2023-10-15 17:24:02,321][52833] Updated weights for policy 0, policy_version 65320 (0.0008) -[2023-10-15 17:24:02,534][52866] Updated weights for policy 1, policy_version 65560 (0.0007) -[2023-10-15 17:24:02,693][52833] Updated weights for policy 0, policy_version 65330 (0.0008) -[2023-10-15 17:24:03,061][52833] Updated weights for policy 0, policy_version 65340 (0.0010) -[2023-10-15 17:24:03,441][51532] Fps is (10 sec: 19660.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 134053888. Throughput: 0: 1778.3, 1: 1785.0. Samples: 33511782. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 17:24:03,442][51532] Avg episode reward: [(0, '59.400'), (1, '76.990')] -[2023-10-15 17:24:03,443][52518] Saving new best policy, reward=76.990! -[2023-10-15 17:24:06,338][52866] Updated weights for policy 1, policy_version 65570 (0.0007) -[2023-10-15 17:24:06,701][52866] Updated weights for policy 1, policy_version 65580 (0.0009) -[2023-10-15 17:24:06,793][52833] Updated weights for policy 0, policy_version 65350 (0.0009) -[2023-10-15 17:24:07,061][52866] Updated weights for policy 1, policy_version 65590 (0.0007) -[2023-10-15 17:24:07,163][52833] Updated weights for policy 0, policy_version 65360 (0.0010) -[2023-10-15 17:24:07,426][52866] Updated weights for policy 1, policy_version 65600 (0.0007) -[2023-10-15 17:24:07,526][52833] Updated weights for policy 0, policy_version 65370 (0.0010) -[2023-10-15 17:24:08,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 134119424. Throughput: 0: 1802.9, 1: 1792.1. Samples: 33532936. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 17:24:08,441][51532] Avg episode reward: [(0, '60.140'), (1, '70.750')] -[2023-10-15 17:24:11,171][52866] Updated weights for policy 1, policy_version 65610 (0.0008) -[2023-10-15 17:24:11,347][52833] Updated weights for policy 0, policy_version 65380 (0.0008) -[2023-10-15 17:24:11,533][52866] Updated weights for policy 1, policy_version 65620 (0.0008) -[2023-10-15 17:24:11,708][52833] Updated weights for policy 0, policy_version 65390 (0.0008) -[2023-10-15 17:24:11,893][52866] Updated weights for policy 1, policy_version 65630 (0.0007) -[2023-10-15 17:24:12,079][52833] Updated weights for policy 0, policy_version 65400 (0.0009) -[2023-10-15 17:24:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 134184960. Throughput: 0: 1782.6, 1: 1782.6. Samples: 33553724. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 17:24:13,441][51532] Avg episode reward: [(0, '61.410'), (1, '68.580')] -[2023-10-15 17:24:15,653][52833] Updated weights for policy 0, policy_version 65410 (0.0007) -[2023-10-15 17:24:15,770][52866] Updated weights for policy 1, policy_version 65640 (0.0008) -[2023-10-15 17:24:16,023][52833] Updated weights for policy 0, policy_version 65420 (0.0009) -[2023-10-15 17:24:16,140][52866] Updated weights for policy 1, policy_version 65650 (0.0008) -[2023-10-15 17:24:16,391][52833] Updated weights for policy 0, policy_version 65430 (0.0010) -[2023-10-15 17:24:16,511][52866] Updated weights for policy 1, policy_version 65660 (0.0008) -[2023-10-15 17:24:16,754][52833] Updated weights for policy 0, policy_version 65440 (0.0010) -[2023-10-15 17:24:18,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 134250496. Throughput: 0: 1801.1, 1: 1806.1. Samples: 33565592. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-15 17:24:18,442][51532] Avg episode reward: [(0, '61.200'), (1, '67.920')] -[2023-10-15 17:24:20,194][52866] Updated weights for policy 1, policy_version 65670 (0.0010) -[2023-10-15 17:24:20,549][52866] Updated weights for policy 1, policy_version 65680 (0.0009) -[2023-10-15 17:24:20,561][52833] Updated weights for policy 0, policy_version 65450 (0.0007) -[2023-10-15 17:24:20,914][52866] Updated weights for policy 1, policy_version 65690 (0.0008) -[2023-10-15 17:24:20,925][52833] Updated weights for policy 0, policy_version 65460 (0.0008) -[2023-10-15 17:24:21,299][52833] Updated weights for policy 0, policy_version 65470 (0.0009) -[2023-10-15 17:24:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 134316032. Throughput: 0: 1784.1, 1: 1783.3. Samples: 33585900. Policy #0 lag: (min: 0.0, avg: 21.6, max: 32.0) -[2023-10-15 17:24:23,442][51532] Avg episode reward: [(0, '66.180'), (1, '66.430')] -[2023-10-15 17:24:24,563][52866] Updated weights for policy 1, policy_version 65700 (0.0009) -[2023-10-15 17:24:24,924][52866] Updated weights for policy 1, policy_version 65710 (0.0009) -[2023-10-15 17:24:25,240][52833] Updated weights for policy 0, policy_version 65480 (0.0009) -[2023-10-15 17:24:25,298][52866] Updated weights for policy 1, policy_version 65720 (0.0007) -[2023-10-15 17:24:25,597][52833] Updated weights for policy 0, policy_version 65490 (0.0009) -[2023-10-15 17:24:25,973][52833] Updated weights for policy 0, policy_version 65500 (0.0009) -[2023-10-15 17:24:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 134381568. Throughput: 0: 1775.1, 1: 1796.1. Samples: 33608404. Policy #0 lag: (min: 0.0, avg: 21.6, max: 32.0) -[2023-10-15 17:24:28,442][51532] Avg episode reward: [(0, '65.170'), (1, '67.790')] -[2023-10-15 17:24:28,910][52866] Updated weights for policy 1, policy_version 65730 (0.0009) -[2023-10-15 17:24:29,287][52866] Updated weights for policy 1, policy_version 65740 (0.0011) -[2023-10-15 17:24:29,602][52833] Updated weights for policy 0, policy_version 65510 (0.0008) -[2023-10-15 17:24:29,647][52866] Updated weights for policy 1, policy_version 65750 (0.0008) -[2023-10-15 17:24:29,970][52833] Updated weights for policy 0, policy_version 65520 (0.0007) -[2023-10-15 17:24:30,012][52866] Updated weights for policy 1, policy_version 65760 (0.0007) -[2023-10-15 17:24:30,350][52833] Updated weights for policy 0, policy_version 65530 (0.0008) -[2023-10-15 17:24:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 134447104. Throughput: 0: 1774.2, 1: 1797.6. Samples: 33618366. Policy #0 lag: (min: 0.0, avg: 21.6, max: 32.0) -[2023-10-15 17:24:33,441][51532] Avg episode reward: [(0, '65.320'), (1, '65.350')] -[2023-10-15 17:24:33,804][52866] Updated weights for policy 1, policy_version 65770 (0.0008) -[2023-10-15 17:24:34,166][52833] Updated weights for policy 0, policy_version 65540 (0.0008) -[2023-10-15 17:24:34,171][52866] Updated weights for policy 1, policy_version 65780 (0.0007) -[2023-10-15 17:24:34,523][52866] Updated weights for policy 1, policy_version 65790 (0.0009) -[2023-10-15 17:24:34,533][52833] Updated weights for policy 0, policy_version 65550 (0.0009) -[2023-10-15 17:24:34,894][52833] Updated weights for policy 0, policy_version 65560 (0.0007) -[2023-10-15 17:24:38,336][52866] Updated weights for policy 1, policy_version 65800 (0.0007) -[2023-10-15 17:24:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 134512640. Throughput: 0: 1781.9, 1: 1789.6. Samples: 33640590. Policy #0 lag: (min: 0.0, avg: 21.6, max: 32.0) -[2023-10-15 17:24:38,442][51532] Avg episode reward: [(0, '63.610'), (1, '65.480')] -[2023-10-15 17:24:38,605][52833] Updated weights for policy 0, policy_version 65570 (0.0007) -[2023-10-15 17:24:38,695][52866] Updated weights for policy 1, policy_version 65810 (0.0007) -[2023-10-15 17:24:38,966][52833] Updated weights for policy 0, policy_version 65580 (0.0008) -[2023-10-15 17:24:39,060][52866] Updated weights for policy 1, policy_version 65820 (0.0009) -[2023-10-15 17:24:39,335][52833] Updated weights for policy 0, policy_version 65590 (0.0007) -[2023-10-15 17:24:39,703][52833] Updated weights for policy 0, policy_version 65600 (0.0007) -[2023-10-15 17:24:42,688][52866] Updated weights for policy 1, policy_version 65830 (0.0008) -[2023-10-15 17:24:43,051][52866] Updated weights for policy 1, policy_version 65840 (0.0007) -[2023-10-15 17:24:43,416][52866] Updated weights for policy 1, policy_version 65850 (0.0009) -[2023-10-15 17:24:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 134578176. Throughput: 0: 1797.2, 1: 1806.6. Samples: 33662820. Policy #0 lag: (min: 0.0, avg: 21.6, max: 32.0) -[2023-10-15 17:24:43,442][51532] Avg episode reward: [(0, '64.720'), (1, '63.530')] -[2023-10-15 17:24:43,463][52833] Updated weights for policy 0, policy_version 65610 (0.0007) -[2023-10-15 17:24:43,836][52833] Updated weights for policy 0, policy_version 65620 (0.0008) -[2023-10-15 17:24:44,198][52833] Updated weights for policy 0, policy_version 65630 (0.0008) -[2023-10-15 17:24:47,119][52866] Updated weights for policy 1, policy_version 65860 (0.0008) -[2023-10-15 17:24:47,483][52866] Updated weights for policy 1, policy_version 65870 (0.0008) -[2023-10-15 17:24:47,848][52866] Updated weights for policy 1, policy_version 65880 (0.0007) -[2023-10-15 17:24:48,181][52833] Updated weights for policy 0, policy_version 65640 (0.0009) -[2023-10-15 17:24:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 134676480. Throughput: 0: 1790.6, 1: 1800.6. Samples: 33673384. Policy #0 lag: (min: 0.0, avg: 21.6, max: 32.0) -[2023-10-15 17:24:48,441][51532] Avg episode reward: [(0, '61.880'), (1, '66.080')] -[2023-10-15 17:24:48,560][52833] Updated weights for policy 0, policy_version 65650 (0.0008) -[2023-10-15 17:24:48,929][52833] Updated weights for policy 0, policy_version 65660 (0.0008) -[2023-10-15 17:24:51,650][52866] Updated weights for policy 1, policy_version 65890 (0.0009) -[2023-10-15 17:24:52,019][52866] Updated weights for policy 1, policy_version 65900 (0.0008) -[2023-10-15 17:24:52,386][52866] Updated weights for policy 1, policy_version 65910 (0.0008) -[2023-10-15 17:24:52,396][52833] Updated weights for policy 0, policy_version 65670 (0.0007) -[2023-10-15 17:24:52,748][52866] Updated weights for policy 1, policy_version 65920 (0.0008) -[2023-10-15 17:24:52,763][52833] Updated weights for policy 0, policy_version 65680 (0.0009) -[2023-10-15 17:24:53,132][52833] Updated weights for policy 0, policy_version 65690 (0.0008) -[2023-10-15 17:24:53,441][51532] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 134774784. Throughput: 0: 1798.9, 1: 1809.6. Samples: 33695320. Policy #0 lag: (min: 0.0, avg: 21.6, max: 32.0) -[2023-10-15 17:24:53,442][51532] Avg episode reward: [(0, '64.570'), (1, '69.400')] -[2023-10-15 17:24:56,622][52866] Updated weights for policy 1, policy_version 65930 (0.0008) -[2023-10-15 17:24:56,984][52866] Updated weights for policy 1, policy_version 65940 (0.0008) -[2023-10-15 17:24:57,025][52833] Updated weights for policy 0, policy_version 65700 (0.0007) -[2023-10-15 17:24:57,344][52866] Updated weights for policy 1, policy_version 65950 (0.0009) -[2023-10-15 17:24:57,390][52833] Updated weights for policy 0, policy_version 65710 (0.0007) -[2023-10-15 17:24:57,758][52833] Updated weights for policy 0, policy_version 65720 (0.0010) -[2023-10-15 17:24:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 134840320. Throughput: 0: 1792.0, 1: 1800.7. Samples: 33715398. Policy #0 lag: (min: 0.0, avg: 21.6, max: 32.0) -[2023-10-15 17:24:58,441][51532] Avg episode reward: [(0, '59.610'), (1, '69.250')] -[2023-10-15 17:25:01,115][52866] Updated weights for policy 1, policy_version 65960 (0.0007) -[2023-10-15 17:25:01,453][52833] Updated weights for policy 0, policy_version 65730 (0.0011) -[2023-10-15 17:25:01,484][52866] Updated weights for policy 1, policy_version 65970 (0.0009) -[2023-10-15 17:25:01,820][52833] Updated weights for policy 0, policy_version 65740 (0.0008) -[2023-10-15 17:25:01,843][52866] Updated weights for policy 1, policy_version 65980 (0.0008) -[2023-10-15 17:25:02,179][52833] Updated weights for policy 0, policy_version 65750 (0.0010) -[2023-10-15 17:25:02,550][52833] Updated weights for policy 0, policy_version 65760 (0.0010) -[2023-10-15 17:25:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134905856. Throughput: 0: 1790.4, 1: 1810.3. Samples: 33727624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:03,442][51532] Avg episode reward: [(0, '60.920'), (1, '67.630')] -[2023-10-15 17:25:05,671][52866] Updated weights for policy 1, policy_version 65990 (0.0008) -[2023-10-15 17:25:06,038][52866] Updated weights for policy 1, policy_version 66000 (0.0007) -[2023-10-15 17:25:06,395][52833] Updated weights for policy 0, policy_version 65770 (0.0008) -[2023-10-15 17:25:06,401][52866] Updated weights for policy 1, policy_version 66010 (0.0008) -[2023-10-15 17:25:06,764][52833] Updated weights for policy 0, policy_version 65780 (0.0009) -[2023-10-15 17:25:07,136][52833] Updated weights for policy 0, policy_version 65790 (0.0007) -[2023-10-15 17:25:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134971392. Throughput: 0: 1794.6, 1: 1802.8. Samples: 33747782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:08,441][51532] Avg episode reward: [(0, '59.860'), (1, '67.170')] -[2023-10-15 17:25:10,046][52866] Updated weights for policy 1, policy_version 66020 (0.0009) -[2023-10-15 17:25:10,420][52866] Updated weights for policy 1, policy_version 66030 (0.0007) -[2023-10-15 17:25:10,779][52866] Updated weights for policy 1, policy_version 66040 (0.0009) -[2023-10-15 17:25:11,099][52833] Updated weights for policy 0, policy_version 65800 (0.0008) -[2023-10-15 17:25:11,470][52833] Updated weights for policy 0, policy_version 65810 (0.0007) -[2023-10-15 17:25:11,843][52833] Updated weights for policy 0, policy_version 65820 (0.0008) -[2023-10-15 17:25:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135036928. Throughput: 0: 1782.7, 1: 1798.3. Samples: 33769548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:13,441][51532] Avg episode reward: [(0, '60.470'), (1, '68.940')] -[2023-10-15 17:25:14,523][52866] Updated weights for policy 1, policy_version 66050 (0.0007) -[2023-10-15 17:25:14,891][52866] Updated weights for policy 1, policy_version 66060 (0.0009) -[2023-10-15 17:25:15,255][52866] Updated weights for policy 1, policy_version 66070 (0.0008) -[2023-10-15 17:25:15,612][52866] Updated weights for policy 1, policy_version 66080 (0.0007) -[2023-10-15 17:25:15,624][52833] Updated weights for policy 0, policy_version 65830 (0.0008) -[2023-10-15 17:25:15,987][52833] Updated weights for policy 0, policy_version 65840 (0.0007) -[2023-10-15 17:25:16,365][52833] Updated weights for policy 0, policy_version 65850 (0.0008) -[2023-10-15 17:25:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 135102464. Throughput: 0: 1799.6, 1: 1792.6. Samples: 33780016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:18,442][51532] Avg episode reward: [(0, '60.400'), (1, '67.200')] -[2023-10-15 17:25:19,402][52866] Updated weights for policy 1, policy_version 66090 (0.0010) -[2023-10-15 17:25:19,771][52866] Updated weights for policy 1, policy_version 66100 (0.0011) -[2023-10-15 17:25:20,137][52866] Updated weights for policy 1, policy_version 66110 (0.0009) -[2023-10-15 17:25:20,212][52833] Updated weights for policy 0, policy_version 65860 (0.0009) -[2023-10-15 17:25:20,579][52833] Updated weights for policy 0, policy_version 65870 (0.0009) -[2023-10-15 17:25:20,948][52833] Updated weights for policy 0, policy_version 65880 (0.0009) -[2023-10-15 17:25:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 135168000. Throughput: 0: 1774.3, 1: 1800.2. Samples: 33801440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:23,442][51532] Avg episode reward: [(0, '56.520'), (1, '67.620')] -[2023-10-15 17:25:23,855][52866] Updated weights for policy 1, policy_version 66120 (0.0009) -[2023-10-15 17:25:24,222][52866] Updated weights for policy 1, policy_version 66130 (0.0007) -[2023-10-15 17:25:24,581][52866] Updated weights for policy 1, policy_version 66140 (0.0007) -[2023-10-15 17:25:24,601][52833] Updated weights for policy 0, policy_version 65890 (0.0009) -[2023-10-15 17:25:24,963][52833] Updated weights for policy 0, policy_version 65900 (0.0007) -[2023-10-15 17:25:25,328][52833] Updated weights for policy 0, policy_version 65910 (0.0010) -[2023-10-15 17:25:25,692][52833] Updated weights for policy 0, policy_version 65920 (0.0010) -[2023-10-15 17:25:28,293][52866] Updated weights for policy 1, policy_version 66150 (0.0009) -[2023-10-15 17:25:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 135233536. Throughput: 0: 1770.7, 1: 1815.3. Samples: 33824190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:28,442][51532] Avg episode reward: [(0, '55.680'), (1, '64.300')] -[2023-10-15 17:25:28,667][52866] Updated weights for policy 1, policy_version 66160 (0.0008) -[2023-10-15 17:25:29,025][52866] Updated weights for policy 1, policy_version 66170 (0.0007) -[2023-10-15 17:25:29,484][52833] Updated weights for policy 0, policy_version 65930 (0.0008) -[2023-10-15 17:25:29,843][52833] Updated weights for policy 0, policy_version 65940 (0.0010) -[2023-10-15 17:25:30,211][52833] Updated weights for policy 0, policy_version 65950 (0.0010) -[2023-10-15 17:25:32,814][52866] Updated weights for policy 1, policy_version 66180 (0.0008) -[2023-10-15 17:25:33,174][52866] Updated weights for policy 1, policy_version 66190 (0.0010) -[2023-10-15 17:25:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 135299072. Throughput: 0: 1768.9, 1: 1800.5. Samples: 33834008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:33,441][51532] Avg episode reward: [(0, '55.140'), (1, '65.470')] -[2023-10-15 17:25:33,533][52866] Updated weights for policy 1, policy_version 66200 (0.0010) -[2023-10-15 17:25:33,898][52833] Updated weights for policy 0, policy_version 65960 (0.0010) -[2023-10-15 17:25:34,262][52833] Updated weights for policy 0, policy_version 65970 (0.0007) -[2023-10-15 17:25:34,627][52833] Updated weights for policy 0, policy_version 65980 (0.0008) -[2023-10-15 17:25:37,325][52866] Updated weights for policy 1, policy_version 66210 (0.0008) -[2023-10-15 17:25:37,687][52866] Updated weights for policy 1, policy_version 66220 (0.0009) -[2023-10-15 17:25:38,054][52866] Updated weights for policy 1, policy_version 66230 (0.0010) -[2023-10-15 17:25:38,358][52833] Updated weights for policy 0, policy_version 65990 (0.0010) -[2023-10-15 17:25:38,418][52866] Updated weights for policy 1, policy_version 66240 (0.0008) -[2023-10-15 17:25:38,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 135397376. Throughput: 0: 1768.8, 1: 1806.7. Samples: 33856220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:38,442][51532] Avg episode reward: [(0, '57.560'), (1, '65.580')] -[2023-10-15 17:25:38,720][52833] Updated weights for policy 0, policy_version 66000 (0.0008) -[2023-10-15 17:25:39,094][52833] Updated weights for policy 0, policy_version 66010 (0.0007) -[2023-10-15 17:25:42,200][52866] Updated weights for policy 1, policy_version 66250 (0.0007) -[2023-10-15 17:25:42,572][52866] Updated weights for policy 1, policy_version 66260 (0.0009) -[2023-10-15 17:25:42,868][52833] Updated weights for policy 0, policy_version 66020 (0.0009) -[2023-10-15 17:25:42,939][52866] Updated weights for policy 1, policy_version 66270 (0.0009) -[2023-10-15 17:25:43,235][52833] Updated weights for policy 0, policy_version 66030 (0.0011) -[2023-10-15 17:25:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 135462912. Throughput: 0: 1791.1, 1: 1795.2. Samples: 33876780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:43,441][51532] Avg episode reward: [(0, '58.320'), (1, '65.290')] -[2023-10-15 17:25:43,448][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000066272_67862528.pth... -[2023-10-15 17:25:43,482][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000064576_66125824.pth -[2023-10-15 17:25:43,608][52833] Updated weights for policy 0, policy_version 66040 (0.0007) -[2023-10-15 17:25:43,904][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000066048_67633152.pth... -[2023-10-15 17:25:43,942][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000064352_65896448.pth -[2023-10-15 17:25:46,755][52866] Updated weights for policy 1, policy_version 66280 (0.0009) -[2023-10-15 17:25:47,123][52866] Updated weights for policy 1, policy_version 66290 (0.0008) -[2023-10-15 17:25:47,409][52833] Updated weights for policy 0, policy_version 66050 (0.0008) -[2023-10-15 17:25:47,485][52866] Updated weights for policy 1, policy_version 66300 (0.0010) -[2023-10-15 17:25:47,783][52833] Updated weights for policy 0, policy_version 66060 (0.0009) -[2023-10-15 17:25:48,158][52833] Updated weights for policy 0, policy_version 66070 (0.0009) -[2023-10-15 17:25:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 135528448. Throughput: 0: 1771.1, 1: 1795.6. Samples: 33888126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:48,441][51532] Avg episode reward: [(0, '59.800'), (1, '59.380')] -[2023-10-15 17:25:48,533][52833] Updated weights for policy 0, policy_version 66080 (0.0011) -[2023-10-15 17:25:51,154][52866] Updated weights for policy 1, policy_version 66310 (0.0010) -[2023-10-15 17:25:51,520][52866] Updated weights for policy 1, policy_version 66320 (0.0009) -[2023-10-15 17:25:51,901][52866] Updated weights for policy 1, policy_version 66330 (0.0008) -[2023-10-15 17:25:52,377][52833] Updated weights for policy 0, policy_version 66090 (0.0008) -[2023-10-15 17:25:52,741][52833] Updated weights for policy 0, policy_version 66100 (0.0007) -[2023-10-15 17:25:53,113][52833] Updated weights for policy 0, policy_version 66110 (0.0007) -[2023-10-15 17:25:53,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135626752. Throughput: 0: 1796.0, 1: 1798.2. Samples: 33909520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:53,441][51532] Avg episode reward: [(0, '60.240'), (1, '59.930')] -[2023-10-15 17:25:55,722][52866] Updated weights for policy 1, policy_version 66340 (0.0008) -[2023-10-15 17:25:56,083][52866] Updated weights for policy 1, policy_version 66350 (0.0011) -[2023-10-15 17:25:56,452][52866] Updated weights for policy 1, policy_version 66360 (0.0008) -[2023-10-15 17:25:56,846][52833] Updated weights for policy 0, policy_version 66120 (0.0009) -[2023-10-15 17:25:57,215][52833] Updated weights for policy 0, policy_version 66130 (0.0007) -[2023-10-15 17:25:57,586][52833] Updated weights for policy 0, policy_version 66140 (0.0008) -[2023-10-15 17:25:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135692288. Throughput: 0: 1784.2, 1: 1789.3. Samples: 33930358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:25:58,441][51532] Avg episode reward: [(0, '63.210'), (1, '57.860')] -[2023-10-15 17:26:00,085][52866] Updated weights for policy 1, policy_version 66370 (0.0008) -[2023-10-15 17:26:00,449][52866] Updated weights for policy 1, policy_version 66380 (0.0007) -[2023-10-15 17:26:00,819][52866] Updated weights for policy 1, policy_version 66390 (0.0007) -[2023-10-15 17:26:01,175][52866] Updated weights for policy 1, policy_version 66400 (0.0008) -[2023-10-15 17:26:01,175][52833] Updated weights for policy 0, policy_version 66150 (0.0009) -[2023-10-15 17:26:01,550][52833] Updated weights for policy 0, policy_version 66160 (0.0009) -[2023-10-15 17:26:01,915][52833] Updated weights for policy 0, policy_version 66170 (0.0007) -[2023-10-15 17:26:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 135757824. Throughput: 0: 1804.6, 1: 1802.7. Samples: 33942344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:26:03,442][51532] Avg episode reward: [(0, '62.150'), (1, '56.650')] -[2023-10-15 17:26:04,878][52866] Updated weights for policy 1, policy_version 66410 (0.0011) -[2023-10-15 17:26:05,247][52866] Updated weights for policy 1, policy_version 66420 (0.0007) -[2023-10-15 17:26:05,545][52833] Updated weights for policy 0, policy_version 66180 (0.0008) -[2023-10-15 17:26:05,609][52866] Updated weights for policy 1, policy_version 66430 (0.0007) -[2023-10-15 17:26:05,912][52833] Updated weights for policy 0, policy_version 66190 (0.0008) -[2023-10-15 17:26:06,278][52833] Updated weights for policy 0, policy_version 66200 (0.0007) -[2023-10-15 17:26:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 135823360. Throughput: 0: 1797.3, 1: 1801.2. Samples: 33963372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:26:08,442][51532] Avg episode reward: [(0, '61.720'), (1, '58.450')] -[2023-10-15 17:26:09,371][52866] Updated weights for policy 1, policy_version 66440 (0.0008) -[2023-10-15 17:26:09,742][52866] Updated weights for policy 1, policy_version 66450 (0.0009) -[2023-10-15 17:26:09,973][52833] Updated weights for policy 0, policy_version 66210 (0.0009) -[2023-10-15 17:26:10,101][52866] Updated weights for policy 1, policy_version 66460 (0.0007) -[2023-10-15 17:26:10,355][52833] Updated weights for policy 0, policy_version 66220 (0.0010) -[2023-10-15 17:26:10,725][52833] Updated weights for policy 0, policy_version 66230 (0.0010) -[2023-10-15 17:26:11,092][52833] Updated weights for policy 0, policy_version 66240 (0.0011) -[2023-10-15 17:26:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 135888896. Throughput: 0: 1794.4, 1: 1792.0. Samples: 33985580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:26:13,443][51532] Avg episode reward: [(0, '61.370'), (1, '56.150')] -[2023-10-15 17:26:13,896][52866] Updated weights for policy 1, policy_version 66470 (0.0009) -[2023-10-15 17:26:14,261][52866] Updated weights for policy 1, policy_version 66480 (0.0008) -[2023-10-15 17:26:14,632][52866] Updated weights for policy 1, policy_version 66490 (0.0009) -[2023-10-15 17:26:14,855][52833] Updated weights for policy 0, policy_version 66250 (0.0007) -[2023-10-15 17:26:15,220][52833] Updated weights for policy 0, policy_version 66260 (0.0008) -[2023-10-15 17:26:15,593][52833] Updated weights for policy 0, policy_version 66270 (0.0008) -[2023-10-15 17:26:18,386][52866] Updated weights for policy 1, policy_version 66500 (0.0008) -[2023-10-15 17:26:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 135954432. Throughput: 0: 1796.5, 1: 1788.3. Samples: 33995322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:26:18,442][51532] Avg episode reward: [(0, '66.040'), (1, '56.190')] -[2023-10-15 17:26:18,746][52866] Updated weights for policy 1, policy_version 66510 (0.0008) -[2023-10-15 17:26:19,120][52866] Updated weights for policy 1, policy_version 66520 (0.0010) -[2023-10-15 17:26:19,444][52833] Updated weights for policy 0, policy_version 66280 (0.0007) -[2023-10-15 17:26:19,814][52833] Updated weights for policy 0, policy_version 66290 (0.0009) -[2023-10-15 17:26:20,184][52833] Updated weights for policy 0, policy_version 66300 (0.0008) -[2023-10-15 17:26:22,904][52866] Updated weights for policy 1, policy_version 66530 (0.0008) -[2023-10-15 17:26:23,267][52866] Updated weights for policy 1, policy_version 66540 (0.0009) -[2023-10-15 17:26:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 136019968. Throughput: 0: 1794.6, 1: 1790.4. Samples: 34017544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:26:23,442][51532] Avg episode reward: [(0, '67.160'), (1, '58.160')] -[2023-10-15 17:26:23,636][52866] Updated weights for policy 1, policy_version 66550 (0.0010) -[2023-10-15 17:26:24,006][52866] Updated weights for policy 1, policy_version 66560 (0.0009) -[2023-10-15 17:26:24,054][52833] Updated weights for policy 0, policy_version 66310 (0.0009) -[2023-10-15 17:26:24,433][52833] Updated weights for policy 0, policy_version 66320 (0.0011) -[2023-10-15 17:26:24,805][52833] Updated weights for policy 0, policy_version 66330 (0.0009) -[2023-10-15 17:26:27,718][52866] Updated weights for policy 1, policy_version 66570 (0.0008) -[2023-10-15 17:26:28,088][52866] Updated weights for policy 1, policy_version 66580 (0.0008) -[2023-10-15 17:26:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 136085504. Throughput: 0: 1799.5, 1: 1804.2. Samples: 34038944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:26:28,441][51532] Avg episode reward: [(0, '67.160'), (1, '57.920')] -[2023-10-15 17:26:28,456][52866] Updated weights for policy 1, policy_version 66590 (0.0009) -[2023-10-15 17:26:28,481][52833] Updated weights for policy 0, policy_version 66340 (0.0009) -[2023-10-15 17:26:28,846][52833] Updated weights for policy 0, policy_version 66350 (0.0010) -[2023-10-15 17:26:29,212][52833] Updated weights for policy 0, policy_version 66360 (0.0010) -[2023-10-15 17:26:32,253][52866] Updated weights for policy 1, policy_version 66600 (0.0010) -[2023-10-15 17:26:32,620][52866] Updated weights for policy 1, policy_version 66610 (0.0009) -[2023-10-15 17:26:32,984][52866] Updated weights for policy 1, policy_version 66620 (0.0009) -[2023-10-15 17:26:33,018][52833] Updated weights for policy 0, policy_version 66370 (0.0009) -[2023-10-15 17:26:33,387][52833] Updated weights for policy 0, policy_version 66380 (0.0008) -[2023-10-15 17:26:33,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 136183808. Throughput: 0: 1792.7, 1: 1794.2. Samples: 34049536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:26:33,442][51532] Avg episode reward: [(0, '68.640'), (1, '56.870')] -[2023-10-15 17:26:33,763][52833] Updated weights for policy 0, policy_version 66390 (0.0009) -[2023-10-15 17:26:34,124][52410] Saving new best policy, reward=68.640! -[2023-10-15 17:26:34,126][52833] Updated weights for policy 0, policy_version 66400 (0.0007) -[2023-10-15 17:26:36,707][52866] Updated weights for policy 1, policy_version 66630 (0.0011) -[2023-10-15 17:26:37,078][52866] Updated weights for policy 1, policy_version 66640 (0.0009) -[2023-10-15 17:26:37,445][52866] Updated weights for policy 1, policy_version 66650 (0.0009) -[2023-10-15 17:26:37,990][52833] Updated weights for policy 0, policy_version 66410 (0.0007) -[2023-10-15 17:26:38,363][52833] Updated weights for policy 0, policy_version 66420 (0.0007) -[2023-10-15 17:26:38,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 136249344. Throughput: 0: 1787.1, 1: 1806.9. Samples: 34071252. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:26:38,442][51532] Avg episode reward: [(0, '66.990'), (1, '58.110')] -[2023-10-15 17:26:38,733][52833] Updated weights for policy 0, policy_version 66430 (0.0008) -[2023-10-15 17:26:41,357][52866] Updated weights for policy 1, policy_version 66660 (0.0008) -[2023-10-15 17:26:41,726][52866] Updated weights for policy 1, policy_version 66670 (0.0009) -[2023-10-15 17:26:42,099][52866] Updated weights for policy 1, policy_version 66680 (0.0008) -[2023-10-15 17:26:42,487][52833] Updated weights for policy 0, policy_version 66440 (0.0007) -[2023-10-15 17:26:42,855][52833] Updated weights for policy 0, policy_version 66450 (0.0007) -[2023-10-15 17:26:43,219][52833] Updated weights for policy 0, policy_version 66460 (0.0008) -[2023-10-15 17:26:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 136347648. Throughput: 0: 1800.1, 1: 1790.6. Samples: 34091942. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:26:43,442][51532] Avg episode reward: [(0, '63.840'), (1, '60.790')] -[2023-10-15 17:26:45,747][52866] Updated weights for policy 1, policy_version 66690 (0.0008) -[2023-10-15 17:26:46,114][52866] Updated weights for policy 1, policy_version 66700 (0.0008) -[2023-10-15 17:26:46,481][52866] Updated weights for policy 1, policy_version 66710 (0.0009) -[2023-10-15 17:26:46,848][52866] Updated weights for policy 1, policy_version 66720 (0.0010) -[2023-10-15 17:26:46,951][52833] Updated weights for policy 0, policy_version 66470 (0.0008) -[2023-10-15 17:26:47,311][52833] Updated weights for policy 0, policy_version 66480 (0.0009) -[2023-10-15 17:26:47,680][52833] Updated weights for policy 0, policy_version 66490 (0.0008) -[2023-10-15 17:26:48,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 136413184. Throughput: 0: 1780.9, 1: 1804.7. Samples: 34103692. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:26:48,441][51532] Avg episode reward: [(0, '67.340'), (1, '60.400')] -[2023-10-15 17:26:50,539][52866] Updated weights for policy 1, policy_version 66730 (0.0007) -[2023-10-15 17:26:50,910][52866] Updated weights for policy 1, policy_version 66740 (0.0008) -[2023-10-15 17:26:51,276][52866] Updated weights for policy 1, policy_version 66750 (0.0009) -[2023-10-15 17:26:51,427][52833] Updated weights for policy 0, policy_version 66500 (0.0008) -[2023-10-15 17:26:51,802][52833] Updated weights for policy 0, policy_version 66510 (0.0007) -[2023-10-15 17:26:52,174][52833] Updated weights for policy 0, policy_version 66520 (0.0007) -[2023-10-15 17:26:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 136478720. Throughput: 0: 1795.9, 1: 1782.5. Samples: 34124398. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:26:53,442][51532] Avg episode reward: [(0, '59.200'), (1, '61.940')] -[2023-10-15 17:26:54,988][52866] Updated weights for policy 1, policy_version 66760 (0.0010) -[2023-10-15 17:26:55,359][52866] Updated weights for policy 1, policy_version 66770 (0.0009) -[2023-10-15 17:26:55,728][52866] Updated weights for policy 1, policy_version 66780 (0.0008) -[2023-10-15 17:26:55,904][52833] Updated weights for policy 0, policy_version 66530 (0.0009) -[2023-10-15 17:26:56,272][52833] Updated weights for policy 0, policy_version 66540 (0.0010) -[2023-10-15 17:26:56,640][52833] Updated weights for policy 0, policy_version 66550 (0.0011) -[2023-10-15 17:26:57,010][52833] Updated weights for policy 0, policy_version 66560 (0.0011) -[2023-10-15 17:26:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 136544256. Throughput: 0: 1780.5, 1: 1789.0. Samples: 34146208. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:26:58,442][51532] Avg episode reward: [(0, '60.180'), (1, '61.040')] -[2023-10-15 17:26:59,339][52866] Updated weights for policy 1, policy_version 66790 (0.0009) -[2023-10-15 17:26:59,709][52866] Updated weights for policy 1, policy_version 66800 (0.0008) -[2023-10-15 17:27:00,078][52866] Updated weights for policy 1, policy_version 66810 (0.0008) -[2023-10-15 17:27:00,711][52833] Updated weights for policy 0, policy_version 66570 (0.0007) -[2023-10-15 17:27:01,087][52833] Updated weights for policy 0, policy_version 66580 (0.0007) -[2023-10-15 17:27:01,449][52833] Updated weights for policy 0, policy_version 66590 (0.0009) -[2023-10-15 17:27:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 136609792. Throughput: 0: 1801.4, 1: 1794.4. Samples: 34157130. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:27:03,442][51532] Avg episode reward: [(0, '62.390'), (1, '60.220')] -[2023-10-15 17:27:03,752][52866] Updated weights for policy 1, policy_version 66820 (0.0008) -[2023-10-15 17:27:04,112][52866] Updated weights for policy 1, policy_version 66830 (0.0007) -[2023-10-15 17:27:04,485][52866] Updated weights for policy 1, policy_version 66840 (0.0009) -[2023-10-15 17:27:05,176][52833] Updated weights for policy 0, policy_version 66600 (0.0010) -[2023-10-15 17:27:05,550][52833] Updated weights for policy 0, policy_version 66610 (0.0009) -[2023-10-15 17:27:05,926][52833] Updated weights for policy 0, policy_version 66620 (0.0009) -[2023-10-15 17:27:08,167][52866] Updated weights for policy 1, policy_version 66850 (0.0009) -[2023-10-15 17:27:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 136675328. Throughput: 0: 1786.7, 1: 1806.5. Samples: 34179236. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:27:08,442][51532] Avg episode reward: [(0, '59.330'), (1, '61.050')] -[2023-10-15 17:27:08,532][52866] Updated weights for policy 1, policy_version 66860 (0.0008) -[2023-10-15 17:27:08,903][52866] Updated weights for policy 1, policy_version 66870 (0.0009) -[2023-10-15 17:27:09,257][52866] Updated weights for policy 1, policy_version 66880 (0.0008) -[2023-10-15 17:27:09,734][52833] Updated weights for policy 0, policy_version 66630 (0.0007) -[2023-10-15 17:27:10,126][52833] Updated weights for policy 0, policy_version 66640 (0.0009) -[2023-10-15 17:27:10,502][52833] Updated weights for policy 0, policy_version 66650 (0.0009) -[2023-10-15 17:27:13,124][52866] Updated weights for policy 1, policy_version 66890 (0.0007) -[2023-10-15 17:27:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 136740864. Throughput: 0: 1792.1, 1: 1816.7. Samples: 34201340. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:27:13,442][51532] Avg episode reward: [(0, '59.110'), (1, '60.860')] -[2023-10-15 17:27:13,490][52866] Updated weights for policy 1, policy_version 66900 (0.0008) -[2023-10-15 17:27:13,860][52866] Updated weights for policy 1, policy_version 66910 (0.0009) -[2023-10-15 17:27:14,195][52833] Updated weights for policy 0, policy_version 66660 (0.0009) -[2023-10-15 17:27:14,574][52833] Updated weights for policy 0, policy_version 66670 (0.0011) -[2023-10-15 17:27:14,945][52833] Updated weights for policy 0, policy_version 66680 (0.0010) -[2023-10-15 17:27:17,527][52866] Updated weights for policy 1, policy_version 66920 (0.0009) -[2023-10-15 17:27:17,906][52866] Updated weights for policy 1, policy_version 66930 (0.0007) -[2023-10-15 17:27:18,268][52866] Updated weights for policy 1, policy_version 66940 (0.0008) -[2023-10-15 17:27:18,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 136839168. Throughput: 0: 1791.9, 1: 1807.3. Samples: 34211500. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-15 17:27:18,441][51532] Avg episode reward: [(0, '59.440'), (1, '57.150')] -[2023-10-15 17:27:18,849][52833] Updated weights for policy 0, policy_version 66690 (0.0008) -[2023-10-15 17:27:19,231][52833] Updated weights for policy 0, policy_version 66700 (0.0009) -[2023-10-15 17:27:19,597][52833] Updated weights for policy 0, policy_version 66710 (0.0011) -[2023-10-15 17:27:19,975][52833] Updated weights for policy 0, policy_version 66720 (0.0010) -[2023-10-15 17:27:22,069][52866] Updated weights for policy 1, policy_version 66950 (0.0007) -[2023-10-15 17:27:22,424][52866] Updated weights for policy 1, policy_version 66960 (0.0007) -[2023-10-15 17:27:22,783][52866] Updated weights for policy 1, policy_version 66970 (0.0007) -[2023-10-15 17:27:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 136904704. Throughput: 0: 1790.9, 1: 1814.2. Samples: 34233482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:27:23,441][51532] Avg episode reward: [(0, '59.040'), (1, '56.220')] -[2023-10-15 17:27:23,777][52833] Updated weights for policy 0, policy_version 66730 (0.0008) -[2023-10-15 17:27:24,155][52833] Updated weights for policy 0, policy_version 66740 (0.0009) -[2023-10-15 17:27:24,529][52833] Updated weights for policy 0, policy_version 66750 (0.0009) -[2023-10-15 17:27:26,380][52866] Updated weights for policy 1, policy_version 66980 (0.0007) -[2023-10-15 17:27:26,739][52866] Updated weights for policy 1, policy_version 66990 (0.0011) -[2023-10-15 17:27:27,117][52866] Updated weights for policy 1, policy_version 67000 (0.0010) -[2023-10-15 17:27:28,295][52833] Updated weights for policy 0, policy_version 66760 (0.0008) -[2023-10-15 17:27:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 136970240. Throughput: 0: 1803.6, 1: 1809.7. Samples: 34254542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:27:28,441][51532] Avg episode reward: [(0, '57.580'), (1, '57.010')] -[2023-10-15 17:27:28,667][52833] Updated weights for policy 0, policy_version 66770 (0.0008) -[2023-10-15 17:27:29,030][52833] Updated weights for policy 0, policy_version 66780 (0.0007) -[2023-10-15 17:27:30,877][52866] Updated weights for policy 1, policy_version 67010 (0.0009) -[2023-10-15 17:27:31,246][52866] Updated weights for policy 1, policy_version 67020 (0.0011) -[2023-10-15 17:27:31,622][52866] Updated weights for policy 1, policy_version 67030 (0.0010) -[2023-10-15 17:27:31,976][52866] Updated weights for policy 1, policy_version 67040 (0.0007) -[2023-10-15 17:27:32,677][52833] Updated weights for policy 0, policy_version 66790 (0.0008) -[2023-10-15 17:27:33,045][52833] Updated weights for policy 0, policy_version 66800 (0.0007) -[2023-10-15 17:27:33,413][52833] Updated weights for policy 0, policy_version 66810 (0.0008) -[2023-10-15 17:27:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 137035776. Throughput: 0: 1782.8, 1: 1817.2. Samples: 34265692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:27:33,441][51532] Avg episode reward: [(0, '58.130'), (1, '54.360')] -[2023-10-15 17:27:35,673][52866] Updated weights for policy 1, policy_version 67050 (0.0008) -[2023-10-15 17:27:36,039][52866] Updated weights for policy 1, policy_version 67060 (0.0008) -[2023-10-15 17:27:36,404][52866] Updated weights for policy 1, policy_version 67070 (0.0008) -[2023-10-15 17:27:37,119][52833] Updated weights for policy 0, policy_version 66820 (0.0009) -[2023-10-15 17:27:37,490][52833] Updated weights for policy 0, policy_version 66830 (0.0010) -[2023-10-15 17:27:37,852][52833] Updated weights for policy 0, policy_version 66840 (0.0010) -[2023-10-15 17:27:38,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 137134080. Throughput: 0: 1803.2, 1: 1817.9. Samples: 34287352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:27:38,442][51532] Avg episode reward: [(0, '56.130'), (1, '55.150')] -[2023-10-15 17:27:40,201][52866] Updated weights for policy 1, policy_version 67080 (0.0011) -[2023-10-15 17:27:40,579][52866] Updated weights for policy 1, policy_version 67090 (0.0009) -[2023-10-15 17:27:40,943][52866] Updated weights for policy 1, policy_version 67100 (0.0008) -[2023-10-15 17:27:41,659][52833] Updated weights for policy 0, policy_version 66850 (0.0007) -[2023-10-15 17:27:42,035][52833] Updated weights for policy 0, policy_version 66860 (0.0007) -[2023-10-15 17:27:42,396][52833] Updated weights for policy 0, policy_version 66870 (0.0009) -[2023-10-15 17:27:42,768][52833] Updated weights for policy 0, policy_version 66880 (0.0007) -[2023-10-15 17:27:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137199616. Throughput: 0: 1786.5, 1: 1815.8. Samples: 34308310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:27:43,442][51532] Avg episode reward: [(0, '56.680'), (1, '54.950')] -[2023-10-15 17:27:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000066880_68485120.pth... -[2023-10-15 17:27:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000067104_68714496.pth... -[2023-10-15 17:27:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000065440_67010560.pth -[2023-10-15 17:27:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000065216_66781184.pth -[2023-10-15 17:27:43,491][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000067104_68714496.pth -[2023-10-15 17:27:43,497][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000066880_68485120.pth -[2023-10-15 17:27:44,761][52866] Updated weights for policy 1, policy_version 67110 (0.0010) -[2023-10-15 17:27:45,136][52866] Updated weights for policy 1, policy_version 67120 (0.0010) -[2023-10-15 17:27:45,496][52866] Updated weights for policy 1, policy_version 67130 (0.0011) -[2023-10-15 17:27:46,428][52833] Updated weights for policy 0, policy_version 66890 (0.0009) -[2023-10-15 17:27:46,794][52833] Updated weights for policy 0, policy_version 66900 (0.0011) -[2023-10-15 17:27:47,167][52833] Updated weights for policy 0, policy_version 66910 (0.0008) -[2023-10-15 17:27:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 137265152. Throughput: 0: 1800.7, 1: 1808.4. Samples: 34319536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:27:48,442][51532] Avg episode reward: [(0, '59.300'), (1, '55.140')] -[2023-10-15 17:27:49,380][52866] Updated weights for policy 1, policy_version 67140 (0.0009) -[2023-10-15 17:27:49,741][52866] Updated weights for policy 1, policy_version 67150 (0.0008) -[2023-10-15 17:27:50,110][52866] Updated weights for policy 1, policy_version 67160 (0.0009) -[2023-10-15 17:27:51,046][52833] Updated weights for policy 0, policy_version 66920 (0.0008) -[2023-10-15 17:27:51,417][52833] Updated weights for policy 0, policy_version 66930 (0.0009) -[2023-10-15 17:27:51,784][52833] Updated weights for policy 0, policy_version 66940 (0.0007) -[2023-10-15 17:27:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137330688. Throughput: 0: 1786.1, 1: 1797.3. Samples: 34340486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:27:53,441][51532] Avg episode reward: [(0, '60.750'), (1, '57.590')] -[2023-10-15 17:27:53,707][52866] Updated weights for policy 1, policy_version 67170 (0.0007) -[2023-10-15 17:27:54,074][52866] Updated weights for policy 1, policy_version 67180 (0.0008) -[2023-10-15 17:27:54,440][52866] Updated weights for policy 1, policy_version 67190 (0.0009) -[2023-10-15 17:27:54,801][52866] Updated weights for policy 1, policy_version 67200 (0.0008) -[2023-10-15 17:27:55,475][52833] Updated weights for policy 0, policy_version 66950 (0.0007) -[2023-10-15 17:27:55,846][52833] Updated weights for policy 0, policy_version 66960 (0.0009) -[2023-10-15 17:27:56,219][52833] Updated weights for policy 0, policy_version 66970 (0.0008) -[2023-10-15 17:27:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 137396224. Throughput: 0: 1780.5, 1: 1812.7. Samples: 34363034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:27:58,441][51532] Avg episode reward: [(0, '60.850'), (1, '56.010')] -[2023-10-15 17:27:58,479][52866] Updated weights for policy 1, policy_version 67210 (0.0008) -[2023-10-15 17:27:58,838][52866] Updated weights for policy 1, policy_version 67220 (0.0007) -[2023-10-15 17:27:59,208][52866] Updated weights for policy 1, policy_version 67230 (0.0007) -[2023-10-15 17:27:59,975][52833] Updated weights for policy 0, policy_version 66980 (0.0008) -[2023-10-15 17:28:00,353][52833] Updated weights for policy 0, policy_version 66990 (0.0009) -[2023-10-15 17:28:00,724][52833] Updated weights for policy 0, policy_version 67000 (0.0008) -[2023-10-15 17:28:02,898][52866] Updated weights for policy 1, policy_version 67240 (0.0010) -[2023-10-15 17:28:03,273][52866] Updated weights for policy 1, policy_version 67250 (0.0010) -[2023-10-15 17:28:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 137461760. Throughput: 0: 1788.9, 1: 1804.8. Samples: 34373220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:28:03,441][51532] Avg episode reward: [(0, '61.920'), (1, '54.430')] -[2023-10-15 17:28:03,641][52866] Updated weights for policy 1, policy_version 67260 (0.0007) -[2023-10-15 17:28:04,398][52833] Updated weights for policy 0, policy_version 67010 (0.0009) -[2023-10-15 17:28:04,777][52833] Updated weights for policy 0, policy_version 67020 (0.0009) -[2023-10-15 17:28:05,148][52833] Updated weights for policy 0, policy_version 67030 (0.0008) -[2023-10-15 17:28:05,519][52833] Updated weights for policy 0, policy_version 67040 (0.0010) -[2023-10-15 17:28:07,357][52866] Updated weights for policy 1, policy_version 67270 (0.0009) -[2023-10-15 17:28:07,726][52866] Updated weights for policy 1, policy_version 67280 (0.0007) -[2023-10-15 17:28:08,098][52866] Updated weights for policy 1, policy_version 67290 (0.0007) -[2023-10-15 17:28:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 137560064. Throughput: 0: 1788.7, 1: 1810.7. Samples: 34395452. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:08,442][51532] Avg episode reward: [(0, '59.640'), (1, '56.360')] -[2023-10-15 17:28:09,476][52833] Updated weights for policy 0, policy_version 67050 (0.0008) -[2023-10-15 17:28:09,840][52833] Updated weights for policy 0, policy_version 67060 (0.0008) -[2023-10-15 17:28:10,218][52833] Updated weights for policy 0, policy_version 67070 (0.0008) -[2023-10-15 17:28:11,777][52866] Updated weights for policy 1, policy_version 67300 (0.0007) -[2023-10-15 17:28:12,154][52866] Updated weights for policy 1, policy_version 67310 (0.0007) -[2023-10-15 17:28:12,513][52866] Updated weights for policy 1, policy_version 67320 (0.0008) -[2023-10-15 17:28:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 137625600. Throughput: 0: 1789.1, 1: 1806.8. Samples: 34416356. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:13,442][51532] Avg episode reward: [(0, '59.200'), (1, '55.250')] -[2023-10-15 17:28:14,134][52833] Updated weights for policy 0, policy_version 67080 (0.0007) -[2023-10-15 17:28:14,501][52833] Updated weights for policy 0, policy_version 67090 (0.0007) -[2023-10-15 17:28:14,867][52833] Updated weights for policy 0, policy_version 67100 (0.0007) -[2023-10-15 17:28:16,420][52866] Updated weights for policy 1, policy_version 67330 (0.0009) -[2023-10-15 17:28:16,780][52866] Updated weights for policy 1, policy_version 67340 (0.0008) -[2023-10-15 17:28:17,142][52866] Updated weights for policy 1, policy_version 67350 (0.0007) -[2023-10-15 17:28:17,505][52866] Updated weights for policy 1, policy_version 67360 (0.0008) -[2023-10-15 17:28:18,381][52833] Updated weights for policy 0, policy_version 67110 (0.0009) -[2023-10-15 17:28:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 137691136. Throughput: 0: 1788.4, 1: 1804.7. Samples: 34427380. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:18,441][51532] Avg episode reward: [(0, '59.740'), (1, '56.290')] -[2023-10-15 17:28:18,759][52833] Updated weights for policy 0, policy_version 67120 (0.0008) -[2023-10-15 17:28:19,127][52833] Updated weights for policy 0, policy_version 67130 (0.0009) -[2023-10-15 17:28:21,172][52866] Updated weights for policy 1, policy_version 67370 (0.0010) -[2023-10-15 17:28:21,536][52866] Updated weights for policy 1, policy_version 67380 (0.0011) -[2023-10-15 17:28:21,906][52866] Updated weights for policy 1, policy_version 67390 (0.0010) -[2023-10-15 17:28:22,881][52833] Updated weights for policy 0, policy_version 67140 (0.0008) -[2023-10-15 17:28:23,258][52833] Updated weights for policy 0, policy_version 67150 (0.0008) -[2023-10-15 17:28:23,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 137756672. Throughput: 0: 1784.5, 1: 1797.3. Samples: 34448530. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:23,441][51532] Avg episode reward: [(0, '60.330'), (1, '61.590')] -[2023-10-15 17:28:23,627][52833] Updated weights for policy 0, policy_version 67160 (0.0009) -[2023-10-15 17:28:25,546][52866] Updated weights for policy 1, policy_version 67400 (0.0008) -[2023-10-15 17:28:25,917][52866] Updated weights for policy 1, policy_version 67410 (0.0007) -[2023-10-15 17:28:26,287][52866] Updated weights for policy 1, policy_version 67420 (0.0008) -[2023-10-15 17:28:27,290][52833] Updated weights for policy 0, policy_version 67170 (0.0009) -[2023-10-15 17:28:27,664][52833] Updated weights for policy 0, policy_version 67180 (0.0010) -[2023-10-15 17:28:28,028][52833] Updated weights for policy 0, policy_version 67190 (0.0009) -[2023-10-15 17:28:28,398][52833] Updated weights for policy 0, policy_version 67200 (0.0007) -[2023-10-15 17:28:28,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 137854976. Throughput: 0: 1805.2, 1: 1795.2. Samples: 34470328. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:28,442][51532] Avg episode reward: [(0, '62.360'), (1, '65.140')] -[2023-10-15 17:28:30,104][52866] Updated weights for policy 1, policy_version 67430 (0.0010) -[2023-10-15 17:28:30,471][52866] Updated weights for policy 1, policy_version 67440 (0.0010) -[2023-10-15 17:28:30,832][52866] Updated weights for policy 1, policy_version 67450 (0.0010) -[2023-10-15 17:28:32,136][52833] Updated weights for policy 0, policy_version 67210 (0.0009) -[2023-10-15 17:28:32,496][52833] Updated weights for policy 0, policy_version 67220 (0.0008) -[2023-10-15 17:28:32,877][52833] Updated weights for policy 0, policy_version 67230 (0.0009) -[2023-10-15 17:28:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 137920512. Throughput: 0: 1786.5, 1: 1799.5. Samples: 34480906. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:33,441][51532] Avg episode reward: [(0, '62.990'), (1, '65.320')] -[2023-10-15 17:28:34,615][52866] Updated weights for policy 1, policy_version 67460 (0.0009) -[2023-10-15 17:28:34,976][52866] Updated weights for policy 1, policy_version 67470 (0.0007) -[2023-10-15 17:28:35,349][52866] Updated weights for policy 1, policy_version 67480 (0.0009) -[2023-10-15 17:28:36,768][52833] Updated weights for policy 0, policy_version 67240 (0.0010) -[2023-10-15 17:28:37,139][52833] Updated weights for policy 0, policy_version 67250 (0.0009) -[2023-10-15 17:28:37,507][52833] Updated weights for policy 0, policy_version 67260 (0.0009) -[2023-10-15 17:28:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137986048. Throughput: 0: 1804.4, 1: 1800.0. Samples: 34502688. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:38,442][51532] Avg episode reward: [(0, '62.690'), (1, '65.620')] -[2023-10-15 17:28:39,163][52866] Updated weights for policy 1, policy_version 67490 (0.0009) -[2023-10-15 17:28:39,532][52866] Updated weights for policy 1, policy_version 67500 (0.0008) -[2023-10-15 17:28:39,893][52866] Updated weights for policy 1, policy_version 67510 (0.0007) -[2023-10-15 17:28:40,266][52866] Updated weights for policy 1, policy_version 67520 (0.0007) -[2023-10-15 17:28:41,134][52833] Updated weights for policy 0, policy_version 67270 (0.0011) -[2023-10-15 17:28:41,523][52833] Updated weights for policy 0, policy_version 67280 (0.0008) -[2023-10-15 17:28:41,884][52833] Updated weights for policy 0, policy_version 67290 (0.0011) -[2023-10-15 17:28:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138051584. Throughput: 0: 1785.9, 1: 1799.6. Samples: 34524382. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:43,441][51532] Avg episode reward: [(0, '59.870'), (1, '67.250')] -[2023-10-15 17:28:43,792][52866] Updated weights for policy 1, policy_version 67530 (0.0007) -[2023-10-15 17:28:44,168][52866] Updated weights for policy 1, policy_version 67540 (0.0008) -[2023-10-15 17:28:44,527][52866] Updated weights for policy 1, policy_version 67550 (0.0008) -[2023-10-15 17:28:45,715][52833] Updated weights for policy 0, policy_version 67300 (0.0011) -[2023-10-15 17:28:46,089][52833] Updated weights for policy 0, policy_version 67310 (0.0009) -[2023-10-15 17:28:46,452][52833] Updated weights for policy 0, policy_version 67320 (0.0008) -[2023-10-15 17:28:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138117120. Throughput: 0: 1803.2, 1: 1799.3. Samples: 34535334. Policy #0 lag: (min: 10.0, avg: 11.1, max: 32.0) -[2023-10-15 17:28:48,442][51532] Avg episode reward: [(0, '60.780'), (1, '65.720')] -[2023-10-15 17:28:48,459][52866] Updated weights for policy 1, policy_version 67560 (0.0010) -[2023-10-15 17:28:48,827][52866] Updated weights for policy 1, policy_version 67570 (0.0009) -[2023-10-15 17:28:49,199][52866] Updated weights for policy 1, policy_version 67580 (0.0007) -[2023-10-15 17:28:50,336][52833] Updated weights for policy 0, policy_version 67330 (0.0010) -[2023-10-15 17:28:50,702][52833] Updated weights for policy 0, policy_version 67340 (0.0007) -[2023-10-15 17:28:51,072][52833] Updated weights for policy 0, policy_version 67350 (0.0008) -[2023-10-15 17:28:51,440][52833] Updated weights for policy 0, policy_version 67360 (0.0009) -[2023-10-15 17:28:52,916][52866] Updated weights for policy 1, policy_version 67590 (0.0008) -[2023-10-15 17:28:53,288][52866] Updated weights for policy 1, policy_version 67600 (0.0009) -[2023-10-15 17:28:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 138182656. Throughput: 0: 1783.2, 1: 1791.9. Samples: 34556330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:28:53,441][51532] Avg episode reward: [(0, '58.790'), (1, '62.460')] -[2023-10-15 17:28:53,661][52866] Updated weights for policy 1, policy_version 67610 (0.0008) -[2023-10-15 17:28:55,198][52833] Updated weights for policy 0, policy_version 67370 (0.0010) -[2023-10-15 17:28:55,578][52833] Updated weights for policy 0, policy_version 67380 (0.0010) -[2023-10-15 17:28:55,957][52833] Updated weights for policy 0, policy_version 67390 (0.0010) -[2023-10-15 17:28:57,291][52866] Updated weights for policy 1, policy_version 67620 (0.0007) -[2023-10-15 17:28:57,660][52866] Updated weights for policy 1, policy_version 67630 (0.0007) -[2023-10-15 17:28:58,019][52866] Updated weights for policy 1, policy_version 67640 (0.0009) -[2023-10-15 17:28:58,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 138280960. Throughput: 0: 1786.0, 1: 1804.0. Samples: 34577908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:28:58,442][51532] Avg episode reward: [(0, '59.670'), (1, '62.980')] -[2023-10-15 17:28:59,597][52833] Updated weights for policy 0, policy_version 67400 (0.0010) -[2023-10-15 17:28:59,972][52833] Updated weights for policy 0, policy_version 67410 (0.0008) -[2023-10-15 17:29:00,337][52833] Updated weights for policy 0, policy_version 67420 (0.0008) -[2023-10-15 17:29:01,841][52866] Updated weights for policy 1, policy_version 67650 (0.0010) -[2023-10-15 17:29:02,205][52866] Updated weights for policy 1, policy_version 67660 (0.0007) -[2023-10-15 17:29:02,576][52866] Updated weights for policy 1, policy_version 67670 (0.0008) -[2023-10-15 17:29:02,940][52866] Updated weights for policy 1, policy_version 67680 (0.0009) -[2023-10-15 17:29:03,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 138346496. Throughput: 0: 1791.1, 1: 1797.8. Samples: 34588882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:03,442][51532] Avg episode reward: [(0, '62.180'), (1, '65.010')] -[2023-10-15 17:29:04,028][52833] Updated weights for policy 0, policy_version 67430 (0.0007) -[2023-10-15 17:29:04,393][52833] Updated weights for policy 0, policy_version 67440 (0.0007) -[2023-10-15 17:29:04,765][52833] Updated weights for policy 0, policy_version 67450 (0.0009) -[2023-10-15 17:29:06,684][52866] Updated weights for policy 1, policy_version 67690 (0.0009) -[2023-10-15 17:29:07,056][52866] Updated weights for policy 1, policy_version 67700 (0.0008) -[2023-10-15 17:29:07,420][52866] Updated weights for policy 1, policy_version 67710 (0.0009) -[2023-10-15 17:29:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 138412032. Throughput: 0: 1790.6, 1: 1810.3. Samples: 34610570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:08,442][51532] Avg episode reward: [(0, '59.890'), (1, '62.110')] -[2023-10-15 17:29:08,611][52833] Updated weights for policy 0, policy_version 67460 (0.0009) -[2023-10-15 17:29:08,986][52833] Updated weights for policy 0, policy_version 67470 (0.0008) -[2023-10-15 17:29:09,351][52833] Updated weights for policy 0, policy_version 67480 (0.0009) -[2023-10-15 17:29:11,044][52866] Updated weights for policy 1, policy_version 67720 (0.0009) -[2023-10-15 17:29:11,403][52866] Updated weights for policy 1, policy_version 67730 (0.0009) -[2023-10-15 17:29:11,780][52866] Updated weights for policy 1, policy_version 67740 (0.0007) -[2023-10-15 17:29:13,003][52833] Updated weights for policy 0, policy_version 67490 (0.0009) -[2023-10-15 17:29:13,377][52833] Updated weights for policy 0, policy_version 67500 (0.0008) -[2023-10-15 17:29:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 138477568. Throughput: 0: 1806.8, 1: 1799.9. Samples: 34632630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:13,442][51532] Avg episode reward: [(0, '59.720'), (1, '62.300')] -[2023-10-15 17:29:13,748][52833] Updated weights for policy 0, policy_version 67510 (0.0008) -[2023-10-15 17:29:14,111][52833] Updated weights for policy 0, policy_version 67520 (0.0007) -[2023-10-15 17:29:15,609][52866] Updated weights for policy 1, policy_version 67750 (0.0009) -[2023-10-15 17:29:15,967][52866] Updated weights for policy 1, policy_version 67760 (0.0008) -[2023-10-15 17:29:16,336][52866] Updated weights for policy 1, policy_version 67770 (0.0008) -[2023-10-15 17:29:17,971][52833] Updated weights for policy 0, policy_version 67530 (0.0009) -[2023-10-15 17:29:18,332][52833] Updated weights for policy 0, policy_version 67540 (0.0009) -[2023-10-15 17:29:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 138543104. Throughput: 0: 1787.7, 1: 1818.6. Samples: 34643188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:18,441][51532] Avg episode reward: [(0, '59.710'), (1, '61.840')] -[2023-10-15 17:29:18,700][52833] Updated weights for policy 0, policy_version 67550 (0.0010) -[2023-10-15 17:29:20,074][52866] Updated weights for policy 1, policy_version 67780 (0.0007) -[2023-10-15 17:29:20,436][52866] Updated weights for policy 1, policy_version 67790 (0.0008) -[2023-10-15 17:29:20,803][52866] Updated weights for policy 1, policy_version 67800 (0.0007) -[2023-10-15 17:29:22,423][52833] Updated weights for policy 0, policy_version 67560 (0.0009) -[2023-10-15 17:29:22,780][52833] Updated weights for policy 0, policy_version 67570 (0.0011) -[2023-10-15 17:29:23,146][52833] Updated weights for policy 0, policy_version 67580 (0.0011) -[2023-10-15 17:29:23,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 138641408. Throughput: 0: 1807.2, 1: 1798.2. Samples: 34664930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:23,442][51532] Avg episode reward: [(0, '58.850'), (1, '56.590')] -[2023-10-15 17:29:24,420][52866] Updated weights for policy 1, policy_version 67810 (0.0008) -[2023-10-15 17:29:24,777][52866] Updated weights for policy 1, policy_version 67820 (0.0009) -[2023-10-15 17:29:25,140][52866] Updated weights for policy 1, policy_version 67830 (0.0008) -[2023-10-15 17:29:25,504][52866] Updated weights for policy 1, policy_version 67840 (0.0008) -[2023-10-15 17:29:27,035][52833] Updated weights for policy 0, policy_version 67590 (0.0008) -[2023-10-15 17:29:27,415][52833] Updated weights for policy 0, policy_version 67600 (0.0009) -[2023-10-15 17:29:27,781][52833] Updated weights for policy 0, policy_version 67610 (0.0007) -[2023-10-15 17:29:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138706944. Throughput: 0: 1798.0, 1: 1797.4. Samples: 34686174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:28,441][51532] Avg episode reward: [(0, '57.880'), (1, '57.880')] -[2023-10-15 17:29:29,216][52866] Updated weights for policy 1, policy_version 67850 (0.0009) -[2023-10-15 17:29:29,572][52866] Updated weights for policy 1, policy_version 67860 (0.0008) -[2023-10-15 17:29:29,937][52866] Updated weights for policy 1, policy_version 67870 (0.0007) -[2023-10-15 17:29:31,384][52833] Updated weights for policy 0, policy_version 67620 (0.0008) -[2023-10-15 17:29:31,745][52833] Updated weights for policy 0, policy_version 67630 (0.0009) -[2023-10-15 17:29:32,118][52833] Updated weights for policy 0, policy_version 67640 (0.0007) -[2023-10-15 17:29:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 138772480. Throughput: 0: 1803.6, 1: 1796.9. Samples: 34697358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:33,442][51532] Avg episode reward: [(0, '56.580'), (1, '53.480')] -[2023-10-15 17:29:33,867][52866] Updated weights for policy 1, policy_version 67880 (0.0009) -[2023-10-15 17:29:34,245][52866] Updated weights for policy 1, policy_version 67890 (0.0010) -[2023-10-15 17:29:34,620][52866] Updated weights for policy 1, policy_version 67900 (0.0010) -[2023-10-15 17:29:35,843][52833] Updated weights for policy 0, policy_version 67650 (0.0007) -[2023-10-15 17:29:36,209][52833] Updated weights for policy 0, policy_version 67660 (0.0009) -[2023-10-15 17:29:36,566][52833] Updated weights for policy 0, policy_version 67670 (0.0007) -[2023-10-15 17:29:36,941][52833] Updated weights for policy 0, policy_version 67680 (0.0009) -[2023-10-15 17:29:38,184][52866] Updated weights for policy 1, policy_version 67910 (0.0007) -[2023-10-15 17:29:38,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138838016. Throughput: 0: 1800.7, 1: 1807.1. Samples: 34718682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:38,442][51532] Avg episode reward: [(0, '56.960'), (1, '55.370')] -[2023-10-15 17:29:38,546][52866] Updated weights for policy 1, policy_version 67920 (0.0007) -[2023-10-15 17:29:38,922][52866] Updated weights for policy 1, policy_version 67930 (0.0007) -[2023-10-15 17:29:40,647][52833] Updated weights for policy 0, policy_version 67690 (0.0008) -[2023-10-15 17:29:41,016][52833] Updated weights for policy 0, policy_version 67700 (0.0008) -[2023-10-15 17:29:41,384][52833] Updated weights for policy 0, policy_version 67710 (0.0010) -[2023-10-15 17:29:42,667][52866] Updated weights for policy 1, policy_version 67940 (0.0007) -[2023-10-15 17:29:43,033][52866] Updated weights for policy 1, policy_version 67950 (0.0007) -[2023-10-15 17:29:43,403][52866] Updated weights for policy 1, policy_version 67960 (0.0007) -[2023-10-15 17:29:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 138903552. Throughput: 0: 1797.2, 1: 1816.7. Samples: 34740532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:43,442][51532] Avg episode reward: [(0, '56.710'), (1, '55.550')] -[2023-10-15 17:29:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000067712_69337088.pth... -[2023-10-15 17:29:43,481][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000066048_67633152.pth -[2023-10-15 17:29:43,697][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000067968_69599232.pth... -[2023-10-15 17:29:43,731][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000066272_67862528.pth -[2023-10-15 17:29:45,094][52833] Updated weights for policy 0, policy_version 67720 (0.0009) -[2023-10-15 17:29:45,464][52833] Updated weights for policy 0, policy_version 67730 (0.0009) -[2023-10-15 17:29:45,833][52833] Updated weights for policy 0, policy_version 67740 (0.0009) -[2023-10-15 17:29:47,169][52866] Updated weights for policy 1, policy_version 67970 (0.0008) -[2023-10-15 17:29:47,538][52866] Updated weights for policy 1, policy_version 67980 (0.0010) -[2023-10-15 17:29:47,903][52866] Updated weights for policy 1, policy_version 67990 (0.0011) -[2023-10-15 17:29:48,272][52866] Updated weights for policy 1, policy_version 68000 (0.0007) -[2023-10-15 17:29:48,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 139001856. Throughput: 0: 1799.5, 1: 1802.4. Samples: 34750966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:48,441][51532] Avg episode reward: [(0, '53.320'), (1, '54.920')] -[2023-10-15 17:29:49,628][52833] Updated weights for policy 0, policy_version 67750 (0.0008) -[2023-10-15 17:29:50,000][52833] Updated weights for policy 0, policy_version 67760 (0.0008) -[2023-10-15 17:29:50,373][52833] Updated weights for policy 0, policy_version 67770 (0.0007) -[2023-10-15 17:29:52,092][52866] Updated weights for policy 1, policy_version 68010 (0.0008) -[2023-10-15 17:29:52,461][52866] Updated weights for policy 1, policy_version 68020 (0.0011) -[2023-10-15 17:29:52,814][52866] Updated weights for policy 1, policy_version 68030 (0.0007) -[2023-10-15 17:29:53,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 139067392. Throughput: 0: 1795.5, 1: 1813.1. Samples: 34772954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:53,441][51532] Avg episode reward: [(0, '54.360'), (1, '57.490')] -[2023-10-15 17:29:53,928][52833] Updated weights for policy 0, policy_version 67780 (0.0009) -[2023-10-15 17:29:54,304][52833] Updated weights for policy 0, policy_version 67790 (0.0008) -[2023-10-15 17:29:54,670][52833] Updated weights for policy 0, policy_version 67800 (0.0008) -[2023-10-15 17:29:56,531][52866] Updated weights for policy 1, policy_version 68040 (0.0007) -[2023-10-15 17:29:56,895][52866] Updated weights for policy 1, policy_version 68050 (0.0010) -[2023-10-15 17:29:57,271][52866] Updated weights for policy 1, policy_version 68060 (0.0007) -[2023-10-15 17:29:58,424][52833] Updated weights for policy 0, policy_version 67810 (0.0010) -[2023-10-15 17:29:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 139132928. Throughput: 0: 1800.9, 1: 1799.6. Samples: 34794650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:29:58,441][51532] Avg episode reward: [(0, '53.580'), (1, '58.730')] -[2023-10-15 17:29:58,792][52833] Updated weights for policy 0, policy_version 67820 (0.0007) -[2023-10-15 17:29:59,162][52833] Updated weights for policy 0, policy_version 67830 (0.0008) -[2023-10-15 17:29:59,525][52833] Updated weights for policy 0, policy_version 67840 (0.0009) -[2023-10-15 17:30:01,063][52866] Updated weights for policy 1, policy_version 68070 (0.0010) -[2023-10-15 17:30:01,430][52866] Updated weights for policy 1, policy_version 68080 (0.0008) -[2023-10-15 17:30:01,797][52866] Updated weights for policy 1, policy_version 68090 (0.0007) -[2023-10-15 17:30:03,312][52833] Updated weights for policy 0, policy_version 67850 (0.0011) -[2023-10-15 17:30:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 139198464. Throughput: 0: 1802.2, 1: 1812.4. Samples: 34805846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:03,441][51532] Avg episode reward: [(0, '53.100'), (1, '58.040')] -[2023-10-15 17:30:03,675][52833] Updated weights for policy 0, policy_version 67860 (0.0009) -[2023-10-15 17:30:04,049][52833] Updated weights for policy 0, policy_version 67870 (0.0010) -[2023-10-15 17:30:05,447][52866] Updated weights for policy 1, policy_version 68100 (0.0008) -[2023-10-15 17:30:05,815][52866] Updated weights for policy 1, policy_version 68110 (0.0008) -[2023-10-15 17:30:06,179][52866] Updated weights for policy 1, policy_version 68120 (0.0010) -[2023-10-15 17:30:07,902][52833] Updated weights for policy 0, policy_version 67880 (0.0009) -[2023-10-15 17:30:08,265][52833] Updated weights for policy 0, policy_version 67890 (0.0008) -[2023-10-15 17:30:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 139264000. Throughput: 0: 1797.6, 1: 1807.8. Samples: 34827174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:08,442][51532] Avg episode reward: [(0, '50.180'), (1, '57.660')] -[2023-10-15 17:30:08,651][52833] Updated weights for policy 0, policy_version 67900 (0.0010) -[2023-10-15 17:30:09,753][52866] Updated weights for policy 1, policy_version 68130 (0.0011) -[2023-10-15 17:30:10,122][52866] Updated weights for policy 1, policy_version 68140 (0.0009) -[2023-10-15 17:30:10,496][52866] Updated weights for policy 1, policy_version 68150 (0.0009) -[2023-10-15 17:30:10,865][52866] Updated weights for policy 1, policy_version 68160 (0.0008) -[2023-10-15 17:30:12,396][52833] Updated weights for policy 0, policy_version 67910 (0.0008) -[2023-10-15 17:30:12,782][52833] Updated weights for policy 0, policy_version 67920 (0.0008) -[2023-10-15 17:30:13,156][52833] Updated weights for policy 0, policy_version 67930 (0.0008) -[2023-10-15 17:30:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 139362304. Throughput: 0: 1807.6, 1: 1800.9. Samples: 34848558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:13,442][51532] Avg episode reward: [(0, '51.770'), (1, '56.380')] -[2023-10-15 17:30:14,590][52866] Updated weights for policy 1, policy_version 68170 (0.0008) -[2023-10-15 17:30:14,952][52866] Updated weights for policy 1, policy_version 68180 (0.0007) -[2023-10-15 17:30:15,320][52866] Updated weights for policy 1, policy_version 68190 (0.0007) -[2023-10-15 17:30:16,948][52833] Updated weights for policy 0, policy_version 67940 (0.0009) -[2023-10-15 17:30:17,310][52833] Updated weights for policy 0, policy_version 67950 (0.0009) -[2023-10-15 17:30:17,689][52833] Updated weights for policy 0, policy_version 67960 (0.0008) -[2023-10-15 17:30:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 139427840. Throughput: 0: 1796.0, 1: 1804.2. Samples: 34859366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:18,442][51532] Avg episode reward: [(0, '49.970'), (1, '57.570')] -[2023-10-15 17:30:19,020][52866] Updated weights for policy 1, policy_version 68200 (0.0008) -[2023-10-15 17:30:19,390][52866] Updated weights for policy 1, policy_version 68210 (0.0007) -[2023-10-15 17:30:19,755][52866] Updated weights for policy 1, policy_version 68220 (0.0008) -[2023-10-15 17:30:21,314][52833] Updated weights for policy 0, policy_version 67970 (0.0009) -[2023-10-15 17:30:21,679][52833] Updated weights for policy 0, policy_version 67980 (0.0009) -[2023-10-15 17:30:22,052][52833] Updated weights for policy 0, policy_version 67990 (0.0008) -[2023-10-15 17:30:22,414][52833] Updated weights for policy 0, policy_version 68000 (0.0008) -[2023-10-15 17:30:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139493376. Throughput: 0: 1804.6, 1: 1802.5. Samples: 34881002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:23,441][51532] Avg episode reward: [(0, '51.050'), (1, '55.860')] -[2023-10-15 17:30:23,569][52866] Updated weights for policy 1, policy_version 68230 (0.0007) -[2023-10-15 17:30:23,949][52866] Updated weights for policy 1, policy_version 68240 (0.0009) -[2023-10-15 17:30:24,314][52866] Updated weights for policy 1, policy_version 68250 (0.0008) -[2023-10-15 17:30:26,175][52833] Updated weights for policy 0, policy_version 68010 (0.0008) -[2023-10-15 17:30:26,547][52833] Updated weights for policy 0, policy_version 68020 (0.0008) -[2023-10-15 17:30:26,917][52833] Updated weights for policy 0, policy_version 68030 (0.0008) -[2023-10-15 17:30:28,106][52866] Updated weights for policy 1, policy_version 68260 (0.0009) -[2023-10-15 17:30:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139558912. Throughput: 0: 1796.4, 1: 1810.5. Samples: 34902840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:28,441][51532] Avg episode reward: [(0, '48.350'), (1, '59.170')] -[2023-10-15 17:30:28,472][52866] Updated weights for policy 1, policy_version 68270 (0.0011) -[2023-10-15 17:30:28,834][52866] Updated weights for policy 1, policy_version 68280 (0.0008) -[2023-10-15 17:30:30,569][52833] Updated weights for policy 0, policy_version 68040 (0.0008) -[2023-10-15 17:30:30,929][52833] Updated weights for policy 0, policy_version 68050 (0.0009) -[2023-10-15 17:30:31,304][52833] Updated weights for policy 0, policy_version 68060 (0.0010) -[2023-10-15 17:30:32,467][52866] Updated weights for policy 1, policy_version 68290 (0.0009) -[2023-10-15 17:30:32,843][52866] Updated weights for policy 1, policy_version 68300 (0.0009) -[2023-10-15 17:30:33,212][52866] Updated weights for policy 1, policy_version 68310 (0.0008) -[2023-10-15 17:30:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 139624448. Throughput: 0: 1812.9, 1: 1802.4. Samples: 34913652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:33,441][51532] Avg episode reward: [(0, '52.670'), (1, '57.490')] -[2023-10-15 17:30:33,576][52866] Updated weights for policy 1, policy_version 68320 (0.0008) -[2023-10-15 17:30:35,006][52833] Updated weights for policy 0, policy_version 68070 (0.0009) -[2023-10-15 17:30:35,367][52833] Updated weights for policy 0, policy_version 68080 (0.0007) -[2023-10-15 17:30:35,743][52833] Updated weights for policy 0, policy_version 68090 (0.0008) -[2023-10-15 17:30:37,138][52866] Updated weights for policy 1, policy_version 68330 (0.0008) -[2023-10-15 17:30:37,498][52866] Updated weights for policy 1, policy_version 68340 (0.0008) -[2023-10-15 17:30:37,867][52866] Updated weights for policy 1, policy_version 68350 (0.0008) -[2023-10-15 17:30:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 139722752. Throughput: 0: 1794.5, 1: 1806.5. Samples: 34935000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:38,441][51532] Avg episode reward: [(0, '49.740'), (1, '59.260')] -[2023-10-15 17:30:39,475][52833] Updated weights for policy 0, policy_version 68100 (0.0009) -[2023-10-15 17:30:39,841][52833] Updated weights for policy 0, policy_version 68110 (0.0010) -[2023-10-15 17:30:40,210][52833] Updated weights for policy 0, policy_version 68120 (0.0010) -[2023-10-15 17:30:41,760][52866] Updated weights for policy 1, policy_version 68360 (0.0007) -[2023-10-15 17:30:42,132][52866] Updated weights for policy 1, policy_version 68370 (0.0007) -[2023-10-15 17:30:42,503][52866] Updated weights for policy 1, policy_version 68380 (0.0008) -[2023-10-15 17:30:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 139788288. Throughput: 0: 1790.6, 1: 1807.9. Samples: 34956582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:43,442][51532] Avg episode reward: [(0, '50.430'), (1, '57.410')] -[2023-10-15 17:30:44,109][52833] Updated weights for policy 0, policy_version 68130 (0.0010) -[2023-10-15 17:30:44,482][52833] Updated weights for policy 0, policy_version 68140 (0.0009) -[2023-10-15 17:30:44,851][52833] Updated weights for policy 0, policy_version 68150 (0.0008) -[2023-10-15 17:30:45,227][52833] Updated weights for policy 0, policy_version 68160 (0.0010) -[2023-10-15 17:30:46,301][52866] Updated weights for policy 1, policy_version 68390 (0.0009) -[2023-10-15 17:30:46,666][52866] Updated weights for policy 1, policy_version 68400 (0.0010) -[2023-10-15 17:30:47,033][52866] Updated weights for policy 1, policy_version 68410 (0.0009) -[2023-10-15 17:30:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 139853824. Throughput: 0: 1785.8, 1: 1808.8. Samples: 34967600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:48,441][51532] Avg episode reward: [(0, '52.300'), (1, '56.390')] -[2023-10-15 17:30:48,941][52833] Updated weights for policy 0, policy_version 68170 (0.0008) -[2023-10-15 17:30:49,311][52833] Updated weights for policy 0, policy_version 68180 (0.0007) -[2023-10-15 17:30:49,672][52833] Updated weights for policy 0, policy_version 68190 (0.0009) -[2023-10-15 17:30:51,005][52866] Updated weights for policy 1, policy_version 68420 (0.0009) -[2023-10-15 17:30:51,378][52866] Updated weights for policy 1, policy_version 68430 (0.0011) -[2023-10-15 17:30:51,741][52866] Updated weights for policy 1, policy_version 68440 (0.0011) -[2023-10-15 17:30:53,378][52833] Updated weights for policy 0, policy_version 68200 (0.0008) -[2023-10-15 17:30:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 139919360. Throughput: 0: 1785.3, 1: 1800.0. Samples: 34988510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:53,442][51532] Avg episode reward: [(0, '52.750'), (1, '56.150')] -[2023-10-15 17:30:53,742][52833] Updated weights for policy 0, policy_version 68210 (0.0007) -[2023-10-15 17:30:54,104][52833] Updated weights for policy 0, policy_version 68220 (0.0007) -[2023-10-15 17:30:55,432][52866] Updated weights for policy 1, policy_version 68450 (0.0010) -[2023-10-15 17:30:55,787][52866] Updated weights for policy 1, policy_version 68460 (0.0007) -[2023-10-15 17:30:56,156][52866] Updated weights for policy 1, policy_version 68470 (0.0008) -[2023-10-15 17:30:56,518][52866] Updated weights for policy 1, policy_version 68480 (0.0008) -[2023-10-15 17:30:58,090][52833] Updated weights for policy 0, policy_version 68230 (0.0007) -[2023-10-15 17:30:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 139984896. Throughput: 0: 1806.6, 1: 1796.5. Samples: 35010698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:30:58,441][51532] Avg episode reward: [(0, '55.700'), (1, '55.590')] -[2023-10-15 17:30:58,479][52833] Updated weights for policy 0, policy_version 68240 (0.0007) -[2023-10-15 17:30:58,850][52833] Updated weights for policy 0, policy_version 68250 (0.0007) -[2023-10-15 17:31:00,330][52866] Updated weights for policy 1, policy_version 68490 (0.0010) -[2023-10-15 17:31:00,705][52866] Updated weights for policy 1, policy_version 68500 (0.0007) -[2023-10-15 17:31:01,067][52866] Updated weights for policy 1, policy_version 68510 (0.0007) -[2023-10-15 17:31:02,593][52833] Updated weights for policy 0, policy_version 68260 (0.0008) -[2023-10-15 17:31:02,963][52833] Updated weights for policy 0, policy_version 68270 (0.0007) -[2023-10-15 17:31:03,333][52833] Updated weights for policy 0, policy_version 68280 (0.0007) -[2023-10-15 17:31:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 140050432. Throughput: 0: 1784.8, 1: 1795.7. Samples: 35020488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:03,442][51532] Avg episode reward: [(0, '54.170'), (1, '56.880')] -[2023-10-15 17:31:04,677][52866] Updated weights for policy 1, policy_version 68520 (0.0008) -[2023-10-15 17:31:05,055][52866] Updated weights for policy 1, policy_version 68530 (0.0007) -[2023-10-15 17:31:05,426][52866] Updated weights for policy 1, policy_version 68540 (0.0007) -[2023-10-15 17:31:07,191][52833] Updated weights for policy 0, policy_version 68290 (0.0009) -[2023-10-15 17:31:07,564][52833] Updated weights for policy 0, policy_version 68300 (0.0008) -[2023-10-15 17:31:07,938][52833] Updated weights for policy 0, policy_version 68310 (0.0009) -[2023-10-15 17:31:08,299][52833] Updated weights for policy 0, policy_version 68320 (0.0008) -[2023-10-15 17:31:08,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 140148736. Throughput: 0: 1804.0, 1: 1789.5. Samples: 35042712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:08,442][51532] Avg episode reward: [(0, '57.010'), (1, '57.510')] -[2023-10-15 17:31:09,241][52866] Updated weights for policy 1, policy_version 68550 (0.0007) -[2023-10-15 17:31:09,611][52866] Updated weights for policy 1, policy_version 68560 (0.0008) -[2023-10-15 17:31:09,982][52866] Updated weights for policy 1, policy_version 68570 (0.0007) -[2023-10-15 17:31:12,017][52833] Updated weights for policy 0, policy_version 68330 (0.0008) -[2023-10-15 17:31:12,386][52833] Updated weights for policy 0, policy_version 68340 (0.0008) -[2023-10-15 17:31:12,749][52833] Updated weights for policy 0, policy_version 68350 (0.0007) -[2023-10-15 17:31:13,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140214272. Throughput: 0: 1786.3, 1: 1794.8. Samples: 35063990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:13,441][51532] Avg episode reward: [(0, '59.470'), (1, '57.810')] -[2023-10-15 17:31:13,670][52866] Updated weights for policy 1, policy_version 68580 (0.0007) -[2023-10-15 17:31:14,033][52866] Updated weights for policy 1, policy_version 68590 (0.0007) -[2023-10-15 17:31:14,411][52866] Updated weights for policy 1, policy_version 68600 (0.0007) -[2023-10-15 17:31:16,405][52833] Updated weights for policy 0, policy_version 68360 (0.0010) -[2023-10-15 17:31:16,761][52833] Updated weights for policy 0, policy_version 68370 (0.0010) -[2023-10-15 17:31:17,133][52833] Updated weights for policy 0, policy_version 68380 (0.0010) -[2023-10-15 17:31:18,065][52866] Updated weights for policy 1, policy_version 68610 (0.0009) -[2023-10-15 17:31:18,440][52866] Updated weights for policy 1, policy_version 68620 (0.0008) -[2023-10-15 17:31:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140279808. Throughput: 0: 1798.1, 1: 1795.1. Samples: 35075350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:18,442][51532] Avg episode reward: [(0, '58.370'), (1, '54.910')] -[2023-10-15 17:31:18,806][52866] Updated weights for policy 1, policy_version 68630 (0.0009) -[2023-10-15 17:31:19,179][52866] Updated weights for policy 1, policy_version 68640 (0.0010) -[2023-10-15 17:31:20,965][52833] Updated weights for policy 0, policy_version 68390 (0.0008) -[2023-10-15 17:31:21,330][52833] Updated weights for policy 0, policy_version 68400 (0.0010) -[2023-10-15 17:31:21,696][52833] Updated weights for policy 0, policy_version 68410 (0.0007) -[2023-10-15 17:31:22,993][52866] Updated weights for policy 1, policy_version 68650 (0.0007) -[2023-10-15 17:31:23,361][52866] Updated weights for policy 1, policy_version 68660 (0.0007) -[2023-10-15 17:31:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140345344. Throughput: 0: 1789.3, 1: 1794.4. Samples: 35096270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:23,441][51532] Avg episode reward: [(0, '60.560'), (1, '55.480')] -[2023-10-15 17:31:23,734][52866] Updated weights for policy 1, policy_version 68670 (0.0008) -[2023-10-15 17:31:25,294][52833] Updated weights for policy 0, policy_version 68420 (0.0009) -[2023-10-15 17:31:25,668][52833] Updated weights for policy 0, policy_version 68430 (0.0008) -[2023-10-15 17:31:26,035][52833] Updated weights for policy 0, policy_version 68440 (0.0008) -[2023-10-15 17:31:27,519][52866] Updated weights for policy 1, policy_version 68680 (0.0009) -[2023-10-15 17:31:27,886][52866] Updated weights for policy 1, policy_version 68690 (0.0009) -[2023-10-15 17:31:28,250][52866] Updated weights for policy 1, policy_version 68700 (0.0009) -[2023-10-15 17:31:28,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 140443648. Throughput: 0: 1788.5, 1: 1798.4. Samples: 35117994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:28,441][51532] Avg episode reward: [(0, '57.480'), (1, '57.820')] -[2023-10-15 17:31:29,733][52833] Updated weights for policy 0, policy_version 68450 (0.0007) -[2023-10-15 17:31:30,095][52833] Updated weights for policy 0, policy_version 68460 (0.0008) -[2023-10-15 17:31:30,462][52833] Updated weights for policy 0, policy_version 68470 (0.0009) -[2023-10-15 17:31:30,825][52833] Updated weights for policy 0, policy_version 68480 (0.0009) -[2023-10-15 17:31:31,911][52866] Updated weights for policy 1, policy_version 68710 (0.0008) -[2023-10-15 17:31:32,280][52866] Updated weights for policy 1, policy_version 68720 (0.0007) -[2023-10-15 17:31:32,642][52866] Updated weights for policy 1, policy_version 68730 (0.0007) -[2023-10-15 17:31:33,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 140509184. Throughput: 0: 1795.1, 1: 1785.8. Samples: 35128740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:33,442][51532] Avg episode reward: [(0, '58.000'), (1, '55.770')] -[2023-10-15 17:31:34,575][52833] Updated weights for policy 0, policy_version 68490 (0.0009) -[2023-10-15 17:31:34,937][52833] Updated weights for policy 0, policy_version 68500 (0.0009) -[2023-10-15 17:31:35,308][52833] Updated weights for policy 0, policy_version 68510 (0.0007) -[2023-10-15 17:31:36,358][52866] Updated weights for policy 1, policy_version 68740 (0.0007) -[2023-10-15 17:31:36,727][52866] Updated weights for policy 1, policy_version 68750 (0.0007) -[2023-10-15 17:31:37,095][52866] Updated weights for policy 1, policy_version 68760 (0.0008) -[2023-10-15 17:31:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 140574720. Throughput: 0: 1792.4, 1: 1799.6. Samples: 35150148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:38,442][51532] Avg episode reward: [(0, '59.510'), (1, '58.240')] -[2023-10-15 17:31:39,061][52833] Updated weights for policy 0, policy_version 68520 (0.0008) -[2023-10-15 17:31:39,430][52833] Updated weights for policy 0, policy_version 68530 (0.0008) -[2023-10-15 17:31:39,794][52833] Updated weights for policy 0, policy_version 68540 (0.0008) -[2023-10-15 17:31:41,041][52866] Updated weights for policy 1, policy_version 68770 (0.0008) -[2023-10-15 17:31:41,403][52866] Updated weights for policy 1, policy_version 68780 (0.0010) -[2023-10-15 17:31:41,765][52866] Updated weights for policy 1, policy_version 68790 (0.0010) -[2023-10-15 17:31:42,125][52866] Updated weights for policy 1, policy_version 68800 (0.0010) -[2023-10-15 17:31:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 140640256. Throughput: 0: 1798.0, 1: 1787.1. Samples: 35172030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:43,442][51532] Avg episode reward: [(0, '57.540'), (1, '58.930')] -[2023-10-15 17:31:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000068800_70451200.pth... -[2023-10-15 17:31:43,498][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000067104_68714496.pth -[2023-10-15 17:31:43,610][52833] Updated weights for policy 0, policy_version 68550 (0.0010) -[2023-10-15 17:31:43,976][52833] Updated weights for policy 0, policy_version 68560 (0.0010) -[2023-10-15 17:31:44,345][52833] Updated weights for policy 0, policy_version 68570 (0.0007) -[2023-10-15 17:31:44,566][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000068576_70221824.pth... -[2023-10-15 17:31:44,607][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000066880_68485120.pth -[2023-10-15 17:31:45,921][52866] Updated weights for policy 1, policy_version 68810 (0.0007) -[2023-10-15 17:31:46,294][52866] Updated weights for policy 1, policy_version 68820 (0.0008) -[2023-10-15 17:31:46,656][52866] Updated weights for policy 1, policy_version 68830 (0.0007) -[2023-10-15 17:31:48,002][52833] Updated weights for policy 0, policy_version 68580 (0.0009) -[2023-10-15 17:31:48,367][52833] Updated weights for policy 0, policy_version 68590 (0.0008) -[2023-10-15 17:31:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 140705792. Throughput: 0: 1800.7, 1: 1810.3. Samples: 35182982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:48,441][51532] Avg episode reward: [(0, '57.640'), (1, '59.990')] -[2023-10-15 17:31:48,737][52833] Updated weights for policy 0, policy_version 68600 (0.0009) -[2023-10-15 17:31:50,469][52866] Updated weights for policy 1, policy_version 68840 (0.0007) -[2023-10-15 17:31:50,839][52866] Updated weights for policy 1, policy_version 68850 (0.0008) -[2023-10-15 17:31:51,197][52866] Updated weights for policy 1, policy_version 68860 (0.0009) -[2023-10-15 17:31:52,547][52833] Updated weights for policy 0, policy_version 68610 (0.0008) -[2023-10-15 17:31:52,920][52833] Updated weights for policy 0, policy_version 68620 (0.0010) -[2023-10-15 17:31:53,292][52833] Updated weights for policy 0, policy_version 68630 (0.0008) -[2023-10-15 17:31:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 140771328. Throughput: 0: 1798.0, 1: 1794.0. Samples: 35204348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:31:53,441][51532] Avg episode reward: [(0, '57.860'), (1, '61.030')] -[2023-10-15 17:31:53,664][52833] Updated weights for policy 0, policy_version 68640 (0.0007) -[2023-10-15 17:31:54,989][52866] Updated weights for policy 1, policy_version 68870 (0.0009) -[2023-10-15 17:31:55,362][52866] Updated weights for policy 1, policy_version 68880 (0.0009) -[2023-10-15 17:31:55,727][52866] Updated weights for policy 1, policy_version 68890 (0.0008) -[2023-10-15 17:31:57,281][52833] Updated weights for policy 0, policy_version 68650 (0.0007) -[2023-10-15 17:31:57,657][52833] Updated weights for policy 0, policy_version 68660 (0.0008) -[2023-10-15 17:31:58,028][52833] Updated weights for policy 0, policy_version 68670 (0.0008) -[2023-10-15 17:31:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 140869632. Throughput: 0: 1811.4, 1: 1788.8. Samples: 35226000. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:31:58,442][51532] Avg episode reward: [(0, '58.280'), (1, '64.390')] -[2023-10-15 17:31:59,460][52866] Updated weights for policy 1, policy_version 68900 (0.0008) -[2023-10-15 17:31:59,812][52866] Updated weights for policy 1, policy_version 68910 (0.0008) -[2023-10-15 17:32:00,181][52866] Updated weights for policy 1, policy_version 68920 (0.0008) -[2023-10-15 17:32:01,722][52833] Updated weights for policy 0, policy_version 68680 (0.0009) -[2023-10-15 17:32:02,079][52833] Updated weights for policy 0, policy_version 68690 (0.0008) -[2023-10-15 17:32:02,454][52833] Updated weights for policy 0, policy_version 68700 (0.0008) -[2023-10-15 17:32:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 140935168. Throughput: 0: 1804.1, 1: 1787.4. Samples: 35236966. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:32:03,441][51532] Avg episode reward: [(0, '53.760'), (1, '62.980')] -[2023-10-15 17:32:03,903][52866] Updated weights for policy 1, policy_version 68930 (0.0008) -[2023-10-15 17:32:04,276][52866] Updated weights for policy 1, policy_version 68940 (0.0007) -[2023-10-15 17:32:04,632][52866] Updated weights for policy 1, policy_version 68950 (0.0008) -[2023-10-15 17:32:05,000][52866] Updated weights for policy 1, policy_version 68960 (0.0009) -[2023-10-15 17:32:06,324][52833] Updated weights for policy 0, policy_version 68710 (0.0007) -[2023-10-15 17:32:06,695][52833] Updated weights for policy 0, policy_version 68720 (0.0007) -[2023-10-15 17:32:07,052][52833] Updated weights for policy 0, policy_version 68730 (0.0009) -[2023-10-15 17:32:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141000704. Throughput: 0: 1812.0, 1: 1793.4. Samples: 35258514. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:32:08,441][51532] Avg episode reward: [(0, '55.260'), (1, '60.970')] -[2023-10-15 17:32:08,666][52866] Updated weights for policy 1, policy_version 68970 (0.0007) -[2023-10-15 17:32:09,035][52866] Updated weights for policy 1, policy_version 68980 (0.0008) -[2023-10-15 17:32:09,409][52866] Updated weights for policy 1, policy_version 68990 (0.0011) -[2023-10-15 17:32:10,781][52833] Updated weights for policy 0, policy_version 68740 (0.0008) -[2023-10-15 17:32:11,152][52833] Updated weights for policy 0, policy_version 68750 (0.0008) -[2023-10-15 17:32:11,522][52833] Updated weights for policy 0, policy_version 68760 (0.0008) -[2023-10-15 17:32:13,123][52866] Updated weights for policy 1, policy_version 69000 (0.0008) -[2023-10-15 17:32:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 141066240. Throughput: 0: 1798.0, 1: 1815.2. Samples: 35280588. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:32:13,441][51532] Avg episode reward: [(0, '53.180'), (1, '61.540')] -[2023-10-15 17:32:13,486][52866] Updated weights for policy 1, policy_version 69010 (0.0007) -[2023-10-15 17:32:13,842][52866] Updated weights for policy 1, policy_version 69020 (0.0008) -[2023-10-15 17:32:15,175][52833] Updated weights for policy 0, policy_version 68770 (0.0007) -[2023-10-15 17:32:15,543][52833] Updated weights for policy 0, policy_version 68780 (0.0010) -[2023-10-15 17:32:15,912][52833] Updated weights for policy 0, policy_version 68790 (0.0008) -[2023-10-15 17:32:16,284][52833] Updated weights for policy 0, policy_version 68800 (0.0011) -[2023-10-15 17:32:17,374][52866] Updated weights for policy 1, policy_version 69030 (0.0008) -[2023-10-15 17:32:17,739][52866] Updated weights for policy 1, policy_version 69040 (0.0009) -[2023-10-15 17:32:18,113][52866] Updated weights for policy 1, policy_version 69050 (0.0008) -[2023-10-15 17:32:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 141164544. Throughput: 0: 1812.3, 1: 1802.8. Samples: 35291418. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:32:18,441][51532] Avg episode reward: [(0, '54.700'), (1, '65.300')] -[2023-10-15 17:32:20,153][52833] Updated weights for policy 0, policy_version 68810 (0.0009) -[2023-10-15 17:32:20,529][52833] Updated weights for policy 0, policy_version 68820 (0.0007) -[2023-10-15 17:32:20,905][52833] Updated weights for policy 0, policy_version 68830 (0.0008) -[2023-10-15 17:32:21,864][52866] Updated weights for policy 1, policy_version 69060 (0.0007) -[2023-10-15 17:32:22,225][52866] Updated weights for policy 1, policy_version 69070 (0.0008) -[2023-10-15 17:32:22,588][52866] Updated weights for policy 1, policy_version 69080 (0.0011) -[2023-10-15 17:32:23,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 141230080. Throughput: 0: 1799.2, 1: 1815.4. Samples: 35312806. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:32:23,441][51532] Avg episode reward: [(0, '56.360'), (1, '64.050')] -[2023-10-15 17:32:24,451][52833] Updated weights for policy 0, policy_version 68840 (0.0007) -[2023-10-15 17:32:24,821][52833] Updated weights for policy 0, policy_version 68850 (0.0007) -[2023-10-15 17:32:25,182][52833] Updated weights for policy 0, policy_version 68860 (0.0007) -[2023-10-15 17:32:26,164][52866] Updated weights for policy 1, policy_version 69090 (0.0010) -[2023-10-15 17:32:26,536][52866] Updated weights for policy 1, policy_version 69100 (0.0008) -[2023-10-15 17:32:26,903][52866] Updated weights for policy 1, policy_version 69110 (0.0009) -[2023-10-15 17:32:27,279][52866] Updated weights for policy 1, policy_version 69120 (0.0008) -[2023-10-15 17:32:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 141295616. Throughput: 0: 1793.5, 1: 1814.1. Samples: 35334374. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:32:28,442][51532] Avg episode reward: [(0, '56.020'), (1, '64.020')] -[2023-10-15 17:32:28,947][52833] Updated weights for policy 0, policy_version 68870 (0.0008) -[2023-10-15 17:32:29,316][52833] Updated weights for policy 0, policy_version 68880 (0.0007) -[2023-10-15 17:32:29,677][52833] Updated weights for policy 0, policy_version 68890 (0.0011) -[2023-10-15 17:32:30,938][52866] Updated weights for policy 1, policy_version 69130 (0.0007) -[2023-10-15 17:32:31,316][52866] Updated weights for policy 1, policy_version 69140 (0.0009) -[2023-10-15 17:32:31,691][52866] Updated weights for policy 1, policy_version 69150 (0.0007) -[2023-10-15 17:32:33,376][52833] Updated weights for policy 0, policy_version 68900 (0.0009) -[2023-10-15 17:32:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 141361152. Throughput: 0: 1794.6, 1: 1812.3. Samples: 35345290. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:32:33,441][51532] Avg episode reward: [(0, '54.930'), (1, '64.470')] -[2023-10-15 17:32:33,754][52833] Updated weights for policy 0, policy_version 68910 (0.0009) -[2023-10-15 17:32:34,111][52833] Updated weights for policy 0, policy_version 68920 (0.0009) -[2023-10-15 17:32:35,388][52866] Updated weights for policy 1, policy_version 69160 (0.0007) -[2023-10-15 17:32:35,757][52866] Updated weights for policy 1, policy_version 69170 (0.0010) -[2023-10-15 17:32:36,129][52866] Updated weights for policy 1, policy_version 69180 (0.0008) -[2023-10-15 17:32:37,846][52833] Updated weights for policy 0, policy_version 68930 (0.0009) -[2023-10-15 17:32:38,211][52833] Updated weights for policy 0, policy_version 68940 (0.0008) -[2023-10-15 17:32:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 141426688. Throughput: 0: 1799.4, 1: 1810.5. Samples: 35366792. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) -[2023-10-15 17:32:38,442][51532] Avg episode reward: [(0, '55.400'), (1, '65.710')] -[2023-10-15 17:32:38,576][52833] Updated weights for policy 0, policy_version 68950 (0.0007) -[2023-10-15 17:32:38,947][52833] Updated weights for policy 0, policy_version 68960 (0.0007) -[2023-10-15 17:32:40,022][52866] Updated weights for policy 1, policy_version 69190 (0.0010) -[2023-10-15 17:32:40,390][52866] Updated weights for policy 1, policy_version 69200 (0.0007) -[2023-10-15 17:32:40,744][52866] Updated weights for policy 1, policy_version 69210 (0.0007) -[2023-10-15 17:32:42,686][52833] Updated weights for policy 0, policy_version 68970 (0.0008) -[2023-10-15 17:32:43,070][52833] Updated weights for policy 0, policy_version 68980 (0.0008) -[2023-10-15 17:32:43,441][52833] Updated weights for policy 0, policy_version 68990 (0.0008) -[2023-10-15 17:32:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 141492224. Throughput: 0: 1804.3, 1: 1811.2. Samples: 35388696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:32:43,442][51532] Avg episode reward: [(0, '50.330'), (1, '64.030')] -[2023-10-15 17:32:44,422][52866] Updated weights for policy 1, policy_version 69220 (0.0007) -[2023-10-15 17:32:44,792][52866] Updated weights for policy 1, policy_version 69230 (0.0010) -[2023-10-15 17:32:45,167][52866] Updated weights for policy 1, policy_version 69240 (0.0012) -[2023-10-15 17:32:47,218][52833] Updated weights for policy 0, policy_version 69000 (0.0009) -[2023-10-15 17:32:47,585][52833] Updated weights for policy 0, policy_version 69010 (0.0008) -[2023-10-15 17:32:47,955][52833] Updated weights for policy 0, policy_version 69020 (0.0009) -[2023-10-15 17:32:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 141590528. Throughput: 0: 1794.9, 1: 1812.7. Samples: 35399310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:32:48,441][51532] Avg episode reward: [(0, '50.520'), (1, '65.670')] -[2023-10-15 17:32:49,123][52866] Updated weights for policy 1, policy_version 69250 (0.0010) -[2023-10-15 17:32:49,480][52866] Updated weights for policy 1, policy_version 69260 (0.0007) -[2023-10-15 17:32:49,857][52866] Updated weights for policy 1, policy_version 69270 (0.0009) -[2023-10-15 17:32:50,223][52866] Updated weights for policy 1, policy_version 69280 (0.0009) -[2023-10-15 17:32:51,716][52833] Updated weights for policy 0, policy_version 69030 (0.0011) -[2023-10-15 17:32:52,087][52833] Updated weights for policy 0, policy_version 69040 (0.0010) -[2023-10-15 17:32:52,462][52833] Updated weights for policy 0, policy_version 69050 (0.0010) -[2023-10-15 17:32:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 141656064. Throughput: 0: 1807.0, 1: 1804.6. Samples: 35421034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:32:53,442][51532] Avg episode reward: [(0, '51.930'), (1, '67.300')] -[2023-10-15 17:32:54,152][52866] Updated weights for policy 1, policy_version 69290 (0.0008) -[2023-10-15 17:32:54,523][52866] Updated weights for policy 1, policy_version 69300 (0.0007) -[2023-10-15 17:32:54,884][52866] Updated weights for policy 1, policy_version 69310 (0.0008) -[2023-10-15 17:32:56,392][52833] Updated weights for policy 0, policy_version 69060 (0.0007) -[2023-10-15 17:32:56,760][52833] Updated weights for policy 0, policy_version 69070 (0.0008) -[2023-10-15 17:32:57,131][52833] Updated weights for policy 0, policy_version 69080 (0.0011) -[2023-10-15 17:32:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141721600. Throughput: 0: 1792.0, 1: 1803.3. Samples: 35442376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:32:58,442][51532] Avg episode reward: [(0, '53.400'), (1, '66.350')] -[2023-10-15 17:32:58,592][52866] Updated weights for policy 1, policy_version 69320 (0.0008) -[2023-10-15 17:32:58,962][52866] Updated weights for policy 1, policy_version 69330 (0.0007) -[2023-10-15 17:32:59,328][52866] Updated weights for policy 1, policy_version 69340 (0.0009) -[2023-10-15 17:33:00,881][52833] Updated weights for policy 0, policy_version 69090 (0.0008) -[2023-10-15 17:33:01,253][52833] Updated weights for policy 0, policy_version 69100 (0.0012) -[2023-10-15 17:33:01,619][52833] Updated weights for policy 0, policy_version 69110 (0.0010) -[2023-10-15 17:33:01,982][52833] Updated weights for policy 0, policy_version 69120 (0.0010) -[2023-10-15 17:33:03,085][52866] Updated weights for policy 1, policy_version 69350 (0.0008) -[2023-10-15 17:33:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 141787136. Throughput: 0: 1812.2, 1: 1793.0. Samples: 35453650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:33:03,441][51532] Avg episode reward: [(0, '52.810'), (1, '64.960')] -[2023-10-15 17:33:03,456][52866] Updated weights for policy 1, policy_version 69360 (0.0008) -[2023-10-15 17:33:03,818][52866] Updated weights for policy 1, policy_version 69370 (0.0008) -[2023-10-15 17:33:05,704][52833] Updated weights for policy 0, policy_version 69130 (0.0007) -[2023-10-15 17:33:06,068][52833] Updated weights for policy 0, policy_version 69140 (0.0007) -[2023-10-15 17:33:06,443][52833] Updated weights for policy 0, policy_version 69150 (0.0007) -[2023-10-15 17:33:07,408][52866] Updated weights for policy 1, policy_version 69380 (0.0008) -[2023-10-15 17:33:07,770][52866] Updated weights for policy 1, policy_version 69390 (0.0011) -[2023-10-15 17:33:08,132][52866] Updated weights for policy 1, policy_version 69400 (0.0008) -[2023-10-15 17:33:08,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 141885440. Throughput: 0: 1795.9, 1: 1802.8. Samples: 35474744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:33:08,442][51532] Avg episode reward: [(0, '51.930'), (1, '67.940')] -[2023-10-15 17:33:09,954][52833] Updated weights for policy 0, policy_version 69160 (0.0010) -[2023-10-15 17:33:10,324][52833] Updated weights for policy 0, policy_version 69170 (0.0010) -[2023-10-15 17:33:10,694][52833] Updated weights for policy 0, policy_version 69180 (0.0011) -[2023-10-15 17:33:11,859][52866] Updated weights for policy 1, policy_version 69410 (0.0008) -[2023-10-15 17:33:12,232][52866] Updated weights for policy 1, policy_version 69420 (0.0007) -[2023-10-15 17:33:12,589][52866] Updated weights for policy 1, policy_version 69430 (0.0007) -[2023-10-15 17:33:12,963][52866] Updated weights for policy 1, policy_version 69440 (0.0007) -[2023-10-15 17:33:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 141950976. Throughput: 0: 1808.3, 1: 1789.9. Samples: 35496294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:33:13,442][51532] Avg episode reward: [(0, '52.650'), (1, '69.150')] -[2023-10-15 17:33:14,528][52833] Updated weights for policy 0, policy_version 69190 (0.0008) -[2023-10-15 17:33:14,890][52833] Updated weights for policy 0, policy_version 69200 (0.0008) -[2023-10-15 17:33:15,259][52833] Updated weights for policy 0, policy_version 69210 (0.0007) -[2023-10-15 17:33:16,546][52866] Updated weights for policy 1, policy_version 69450 (0.0009) -[2023-10-15 17:33:16,923][52866] Updated weights for policy 1, policy_version 69460 (0.0010) -[2023-10-15 17:33:17,282][52866] Updated weights for policy 1, policy_version 69470 (0.0010) -[2023-10-15 17:33:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142016512. Throughput: 0: 1808.4, 1: 1802.2. Samples: 35507770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:33:18,442][51532] Avg episode reward: [(0, '55.360'), (1, '69.110')] -[2023-10-15 17:33:18,965][52833] Updated weights for policy 0, policy_version 69220 (0.0008) -[2023-10-15 17:33:19,338][52833] Updated weights for policy 0, policy_version 69230 (0.0010) -[2023-10-15 17:33:19,706][52833] Updated weights for policy 0, policy_version 69240 (0.0009) -[2023-10-15 17:33:21,107][52866] Updated weights for policy 1, policy_version 69480 (0.0008) -[2023-10-15 17:33:21,469][52866] Updated weights for policy 1, policy_version 69490 (0.0011) -[2023-10-15 17:33:21,831][52866] Updated weights for policy 1, policy_version 69500 (0.0009) -[2023-10-15 17:33:23,302][52833] Updated weights for policy 0, policy_version 69250 (0.0010) -[2023-10-15 17:33:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 142082048. Throughput: 0: 1804.5, 1: 1794.9. Samples: 35528768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:33:23,441][51532] Avg episode reward: [(0, '56.720'), (1, '66.710')] -[2023-10-15 17:33:23,681][52833] Updated weights for policy 0, policy_version 69260 (0.0007) -[2023-10-15 17:33:24,045][52833] Updated weights for policy 0, policy_version 69270 (0.0010) -[2023-10-15 17:33:24,420][52833] Updated weights for policy 0, policy_version 69280 (0.0007) -[2023-10-15 17:33:25,717][52866] Updated weights for policy 1, policy_version 69510 (0.0010) -[2023-10-15 17:33:26,091][52866] Updated weights for policy 1, policy_version 69520 (0.0011) -[2023-10-15 17:33:26,460][52866] Updated weights for policy 1, policy_version 69530 (0.0010) -[2023-10-15 17:33:28,029][52833] Updated weights for policy 0, policy_version 69290 (0.0007) -[2023-10-15 17:33:28,401][52833] Updated weights for policy 0, policy_version 69300 (0.0007) -[2023-10-15 17:33:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 142147584. Throughput: 0: 1814.9, 1: 1790.3. Samples: 35550930. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:33:28,441][51532] Avg episode reward: [(0, '57.180'), (1, '66.220')] -[2023-10-15 17:33:28,771][52833] Updated weights for policy 0, policy_version 69310 (0.0010) -[2023-10-15 17:33:30,154][52866] Updated weights for policy 1, policy_version 69540 (0.0008) -[2023-10-15 17:33:30,515][52866] Updated weights for policy 1, policy_version 69550 (0.0007) -[2023-10-15 17:33:30,881][52866] Updated weights for policy 1, policy_version 69560 (0.0007) -[2023-10-15 17:33:32,568][52833] Updated weights for policy 0, policy_version 69320 (0.0008) -[2023-10-15 17:33:32,939][52833] Updated weights for policy 0, policy_version 69330 (0.0008) -[2023-10-15 17:33:33,307][52833] Updated weights for policy 0, policy_version 69340 (0.0007) -[2023-10-15 17:33:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 142213120. Throughput: 0: 1804.4, 1: 1801.4. Samples: 35561570. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:33:33,441][51532] Avg episode reward: [(0, '59.040'), (1, '68.690')] -[2023-10-15 17:33:34,626][52866] Updated weights for policy 1, policy_version 69570 (0.0009) -[2023-10-15 17:33:34,990][52866] Updated weights for policy 1, policy_version 69580 (0.0007) -[2023-10-15 17:33:35,352][52866] Updated weights for policy 1, policy_version 69590 (0.0008) -[2023-10-15 17:33:35,714][52866] Updated weights for policy 1, policy_version 69600 (0.0007) -[2023-10-15 17:33:37,046][52833] Updated weights for policy 0, policy_version 69350 (0.0007) -[2023-10-15 17:33:37,414][52833] Updated weights for policy 0, policy_version 69360 (0.0008) -[2023-10-15 17:33:37,792][52833] Updated weights for policy 0, policy_version 69370 (0.0009) -[2023-10-15 17:33:38,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 142311424. Throughput: 0: 1813.2, 1: 1801.0. Samples: 35583674. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:33:38,442][51532] Avg episode reward: [(0, '59.390'), (1, '70.060')] -[2023-10-15 17:33:39,528][52866] Updated weights for policy 1, policy_version 69610 (0.0008) -[2023-10-15 17:33:39,899][52866] Updated weights for policy 1, policy_version 69620 (0.0007) -[2023-10-15 17:33:40,264][52866] Updated weights for policy 1, policy_version 69630 (0.0009) -[2023-10-15 17:33:41,593][52833] Updated weights for policy 0, policy_version 69380 (0.0008) -[2023-10-15 17:33:41,967][52833] Updated weights for policy 0, policy_version 69390 (0.0008) -[2023-10-15 17:33:42,336][52833] Updated weights for policy 0, policy_version 69400 (0.0007) -[2023-10-15 17:33:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 142376960. Throughput: 0: 1808.0, 1: 1795.9. Samples: 35604552. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:33:43,442][51532] Avg episode reward: [(0, '61.640'), (1, '68.510')] -[2023-10-15 17:33:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000069632_71303168.pth... -[2023-10-15 17:33:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000069408_71073792.pth... -[2023-10-15 17:33:43,479][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000067968_69599232.pth -[2023-10-15 17:33:43,487][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000067712_69337088.pth -[2023-10-15 17:33:44,123][52866] Updated weights for policy 1, policy_version 69640 (0.0008) -[2023-10-15 17:33:44,489][52866] Updated weights for policy 1, policy_version 69650 (0.0008) -[2023-10-15 17:33:44,854][52866] Updated weights for policy 1, policy_version 69660 (0.0011) -[2023-10-15 17:33:45,992][52833] Updated weights for policy 0, policy_version 69410 (0.0007) -[2023-10-15 17:33:46,363][52833] Updated weights for policy 0, policy_version 69420 (0.0007) -[2023-10-15 17:33:46,723][52833] Updated weights for policy 0, policy_version 69430 (0.0008) -[2023-10-15 17:33:47,095][52833] Updated weights for policy 0, policy_version 69440 (0.0007) -[2023-10-15 17:33:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142442496. Throughput: 0: 1808.6, 1: 1799.2. Samples: 35616000. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:33:48,441][51532] Avg episode reward: [(0, '61.390'), (1, '72.490')] -[2023-10-15 17:33:48,662][52866] Updated weights for policy 1, policy_version 69670 (0.0010) -[2023-10-15 17:33:49,032][52866] Updated weights for policy 1, policy_version 69680 (0.0008) -[2023-10-15 17:33:49,399][52866] Updated weights for policy 1, policy_version 69690 (0.0007) -[2023-10-15 17:33:50,797][52833] Updated weights for policy 0, policy_version 69450 (0.0010) -[2023-10-15 17:33:51,164][52833] Updated weights for policy 0, policy_version 69460 (0.0010) -[2023-10-15 17:33:51,536][52833] Updated weights for policy 0, policy_version 69470 (0.0008) -[2023-10-15 17:33:52,854][52866] Updated weights for policy 1, policy_version 69700 (0.0008) -[2023-10-15 17:33:53,218][52866] Updated weights for policy 1, policy_version 69710 (0.0009) -[2023-10-15 17:33:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 142508032. Throughput: 0: 1803.4, 1: 1797.8. Samples: 35636798. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:33:53,441][51532] Avg episode reward: [(0, '60.380'), (1, '70.090')] -[2023-10-15 17:33:53,589][52866] Updated weights for policy 1, policy_version 69720 (0.0008) -[2023-10-15 17:33:55,435][52833] Updated weights for policy 0, policy_version 69480 (0.0008) -[2023-10-15 17:33:55,805][52833] Updated weights for policy 0, policy_version 69490 (0.0009) -[2023-10-15 17:33:56,179][52833] Updated weights for policy 0, policy_version 69500 (0.0008) -[2023-10-15 17:33:57,267][52866] Updated weights for policy 1, policy_version 69730 (0.0010) -[2023-10-15 17:33:57,630][52866] Updated weights for policy 1, policy_version 69740 (0.0009) -[2023-10-15 17:33:58,005][52866] Updated weights for policy 1, policy_version 69750 (0.0009) -[2023-10-15 17:33:58,367][52866] Updated weights for policy 1, policy_version 69760 (0.0007) -[2023-10-15 17:33:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 142606336. Throughput: 0: 1795.8, 1: 1810.6. Samples: 35658582. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:33:58,442][51532] Avg episode reward: [(0, '60.830'), (1, '68.190')] -[2023-10-15 17:33:59,854][52833] Updated weights for policy 0, policy_version 69510 (0.0009) -[2023-10-15 17:34:00,240][52833] Updated weights for policy 0, policy_version 69520 (0.0010) -[2023-10-15 17:34:00,608][52833] Updated weights for policy 0, policy_version 69530 (0.0007) -[2023-10-15 17:34:02,152][52866] Updated weights for policy 1, policy_version 69770 (0.0010) -[2023-10-15 17:34:02,513][52866] Updated weights for policy 1, policy_version 69780 (0.0010) -[2023-10-15 17:34:02,879][52866] Updated weights for policy 1, policy_version 69790 (0.0010) -[2023-10-15 17:34:03,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 142671872. Throughput: 0: 1793.2, 1: 1796.6. Samples: 35669312. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:34:03,442][51532] Avg episode reward: [(0, '60.680'), (1, '68.170')] -[2023-10-15 17:34:04,382][52833] Updated weights for policy 0, policy_version 69540 (0.0007) -[2023-10-15 17:34:04,755][52833] Updated weights for policy 0, policy_version 69550 (0.0011) -[2023-10-15 17:34:05,137][52833] Updated weights for policy 0, policy_version 69560 (0.0010) -[2023-10-15 17:34:06,675][52866] Updated weights for policy 1, policy_version 69800 (0.0011) -[2023-10-15 17:34:07,041][52866] Updated weights for policy 1, policy_version 69810 (0.0012) -[2023-10-15 17:34:07,409][52866] Updated weights for policy 1, policy_version 69820 (0.0010) -[2023-10-15 17:34:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142737408. Throughput: 0: 1795.4, 1: 1808.4. Samples: 35690940. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-15 17:34:08,442][51532] Avg episode reward: [(0, '59.270'), (1, '67.120')] -[2023-10-15 17:34:08,923][52833] Updated weights for policy 0, policy_version 69570 (0.0007) -[2023-10-15 17:34:09,293][52833] Updated weights for policy 0, policy_version 69580 (0.0007) -[2023-10-15 17:34:09,661][52833] Updated weights for policy 0, policy_version 69590 (0.0008) -[2023-10-15 17:34:10,036][52833] Updated weights for policy 0, policy_version 69600 (0.0009) -[2023-10-15 17:34:11,482][52866] Updated weights for policy 1, policy_version 69830 (0.0011) -[2023-10-15 17:34:11,872][52866] Updated weights for policy 1, policy_version 69840 (0.0010) -[2023-10-15 17:34:12,246][52866] Updated weights for policy 1, policy_version 69850 (0.0010) -[2023-10-15 17:34:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142802944. Throughput: 0: 1801.5, 1: 1791.2. Samples: 35712604. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:13,442][51532] Avg episode reward: [(0, '57.930'), (1, '66.220')] -[2023-10-15 17:34:13,699][52833] Updated weights for policy 0, policy_version 69610 (0.0009) -[2023-10-15 17:34:14,080][52833] Updated weights for policy 0, policy_version 69620 (0.0009) -[2023-10-15 17:34:14,442][52833] Updated weights for policy 0, policy_version 69630 (0.0008) -[2023-10-15 17:34:15,711][52866] Updated weights for policy 1, policy_version 69860 (0.0009) -[2023-10-15 17:34:16,082][52866] Updated weights for policy 1, policy_version 69870 (0.0007) -[2023-10-15 17:34:16,453][52866] Updated weights for policy 1, policy_version 69880 (0.0008) -[2023-10-15 17:34:18,275][52833] Updated weights for policy 0, policy_version 69640 (0.0008) -[2023-10-15 17:34:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 142868480. Throughput: 0: 1790.5, 1: 1803.4. Samples: 35723298. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:18,442][51532] Avg episode reward: [(0, '57.660'), (1, '67.840')] -[2023-10-15 17:34:18,641][52833] Updated weights for policy 0, policy_version 69650 (0.0008) -[2023-10-15 17:34:19,000][52833] Updated weights for policy 0, policy_version 69660 (0.0009) -[2023-10-15 17:34:20,148][52866] Updated weights for policy 1, policy_version 69890 (0.0008) -[2023-10-15 17:34:20,517][52866] Updated weights for policy 1, policy_version 69900 (0.0007) -[2023-10-15 17:34:20,881][52866] Updated weights for policy 1, policy_version 69910 (0.0008) -[2023-10-15 17:34:21,246][52866] Updated weights for policy 1, policy_version 69920 (0.0008) -[2023-10-15 17:34:22,829][52833] Updated weights for policy 0, policy_version 69670 (0.0010) -[2023-10-15 17:34:23,193][52833] Updated weights for policy 0, policy_version 69680 (0.0008) -[2023-10-15 17:34:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 142934016. Throughput: 0: 1793.8, 1: 1785.6. Samples: 35744746. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:23,441][51532] Avg episode reward: [(0, '60.300'), (1, '67.750')] -[2023-10-15 17:34:23,565][52833] Updated weights for policy 0, policy_version 69690 (0.0008) -[2023-10-15 17:34:24,993][52866] Updated weights for policy 1, policy_version 69930 (0.0011) -[2023-10-15 17:34:25,357][52866] Updated weights for policy 1, policy_version 69940 (0.0010) -[2023-10-15 17:34:25,723][52866] Updated weights for policy 1, policy_version 69950 (0.0010) -[2023-10-15 17:34:27,169][52833] Updated weights for policy 0, policy_version 69700 (0.0008) -[2023-10-15 17:34:27,527][52833] Updated weights for policy 0, policy_version 69710 (0.0008) -[2023-10-15 17:34:27,898][52833] Updated weights for policy 0, policy_version 69720 (0.0007) -[2023-10-15 17:34:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 143032320. Throughput: 0: 1802.3, 1: 1797.8. Samples: 35766556. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:28,442][51532] Avg episode reward: [(0, '61.500'), (1, '69.590')] -[2023-10-15 17:34:29,333][52866] Updated weights for policy 1, policy_version 69960 (0.0010) -[2023-10-15 17:34:29,698][52866] Updated weights for policy 1, policy_version 69970 (0.0011) -[2023-10-15 17:34:30,057][52866] Updated weights for policy 1, policy_version 69980 (0.0010) -[2023-10-15 17:34:31,703][52833] Updated weights for policy 0, policy_version 69730 (0.0008) -[2023-10-15 17:34:32,066][52833] Updated weights for policy 0, policy_version 69740 (0.0007) -[2023-10-15 17:34:32,430][52833] Updated weights for policy 0, policy_version 69750 (0.0010) -[2023-10-15 17:34:32,799][52833] Updated weights for policy 0, policy_version 69760 (0.0010) -[2023-10-15 17:34:33,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 143097856. Throughput: 0: 1789.9, 1: 1799.5. Samples: 35777524. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:33,442][51532] Avg episode reward: [(0, '63.080'), (1, '70.780')] -[2023-10-15 17:34:33,724][52866] Updated weights for policy 1, policy_version 69990 (0.0009) -[2023-10-15 17:34:34,098][52866] Updated weights for policy 1, policy_version 70000 (0.0008) -[2023-10-15 17:34:34,460][52866] Updated weights for policy 1, policy_version 70010 (0.0007) -[2023-10-15 17:34:36,589][52833] Updated weights for policy 0, policy_version 69770 (0.0010) -[2023-10-15 17:34:36,946][52833] Updated weights for policy 0, policy_version 69780 (0.0009) -[2023-10-15 17:34:37,320][52833] Updated weights for policy 0, policy_version 69790 (0.0009) -[2023-10-15 17:34:38,173][52866] Updated weights for policy 1, policy_version 70020 (0.0009) -[2023-10-15 17:34:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143163392. Throughput: 0: 1809.5, 1: 1801.6. Samples: 35799294. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:38,441][51532] Avg episode reward: [(0, '62.510'), (1, '70.470')] -[2023-10-15 17:34:38,535][52866] Updated weights for policy 1, policy_version 70030 (0.0011) -[2023-10-15 17:34:38,909][52866] Updated weights for policy 1, policy_version 70040 (0.0010) -[2023-10-15 17:34:40,920][52833] Updated weights for policy 0, policy_version 69800 (0.0008) -[2023-10-15 17:34:41,281][52833] Updated weights for policy 0, policy_version 69810 (0.0008) -[2023-10-15 17:34:41,653][52833] Updated weights for policy 0, policy_version 69820 (0.0010) -[2023-10-15 17:34:42,664][52866] Updated weights for policy 1, policy_version 70050 (0.0009) -[2023-10-15 17:34:43,017][52866] Updated weights for policy 1, policy_version 70060 (0.0007) -[2023-10-15 17:34:43,380][52866] Updated weights for policy 1, policy_version 70070 (0.0008) -[2023-10-15 17:34:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 143228928. Throughput: 0: 1791.8, 1: 1812.1. Samples: 35820758. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:43,442][51532] Avg episode reward: [(0, '62.820'), (1, '67.050')] -[2023-10-15 17:34:43,747][52866] Updated weights for policy 1, policy_version 70080 (0.0007) -[2023-10-15 17:34:45,421][52833] Updated weights for policy 0, policy_version 69830 (0.0007) -[2023-10-15 17:34:45,793][52833] Updated weights for policy 0, policy_version 69840 (0.0009) -[2023-10-15 17:34:46,155][52833] Updated weights for policy 0, policy_version 69850 (0.0008) -[2023-10-15 17:34:47,525][52866] Updated weights for policy 1, policy_version 70090 (0.0010) -[2023-10-15 17:34:47,895][52866] Updated weights for policy 1, policy_version 70100 (0.0011) -[2023-10-15 17:34:48,259][52866] Updated weights for policy 1, policy_version 70110 (0.0011) -[2023-10-15 17:34:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 143327232. Throughput: 0: 1809.8, 1: 1799.8. Samples: 35831744. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:48,441][51532] Avg episode reward: [(0, '63.360'), (1, '65.530')] -[2023-10-15 17:34:49,969][52833] Updated weights for policy 0, policy_version 69860 (0.0008) -[2023-10-15 17:34:50,340][52833] Updated weights for policy 0, policy_version 69870 (0.0011) -[2023-10-15 17:34:50,709][52833] Updated weights for policy 0, policy_version 69880 (0.0010) -[2023-10-15 17:34:52,210][52866] Updated weights for policy 1, policy_version 70120 (0.0010) -[2023-10-15 17:34:52,577][52866] Updated weights for policy 1, policy_version 70130 (0.0010) -[2023-10-15 17:34:52,937][52866] Updated weights for policy 1, policy_version 70140 (0.0009) -[2023-10-15 17:34:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 143392768. Throughput: 0: 1793.6, 1: 1813.2. Samples: 35853248. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) -[2023-10-15 17:34:53,442][51532] Avg episode reward: [(0, '65.360'), (1, '64.660')] -[2023-10-15 17:34:54,441][52833] Updated weights for policy 0, policy_version 69890 (0.0011) -[2023-10-15 17:34:54,805][52833] Updated weights for policy 0, policy_version 69900 (0.0010) -[2023-10-15 17:34:55,169][52833] Updated weights for policy 0, policy_version 69910 (0.0011) -[2023-10-15 17:34:55,532][52833] Updated weights for policy 0, policy_version 69920 (0.0010) -[2023-10-15 17:34:56,929][52866] Updated weights for policy 1, policy_version 70150 (0.0009) -[2023-10-15 17:34:57,308][52866] Updated weights for policy 1, policy_version 70160 (0.0008) -[2023-10-15 17:34:57,667][52866] Updated weights for policy 1, policy_version 70170 (0.0008) -[2023-10-15 17:34:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143458304. Throughput: 0: 1792.6, 1: 1802.1. Samples: 35874366. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:34:58,442][51532] Avg episode reward: [(0, '66.230'), (1, '67.160')] -[2023-10-15 17:34:59,266][52833] Updated weights for policy 0, policy_version 69930 (0.0010) -[2023-10-15 17:34:59,635][52833] Updated weights for policy 0, policy_version 69940 (0.0010) -[2023-10-15 17:34:59,997][52833] Updated weights for policy 0, policy_version 69950 (0.0007) -[2023-10-15 17:35:01,226][52866] Updated weights for policy 1, policy_version 70180 (0.0009) -[2023-10-15 17:35:01,588][52866] Updated weights for policy 1, policy_version 70190 (0.0009) -[2023-10-15 17:35:01,945][52866] Updated weights for policy 1, policy_version 70200 (0.0008) -[2023-10-15 17:35:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143523840. Throughput: 0: 1800.8, 1: 1812.4. Samples: 35885892. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:35:03,442][51532] Avg episode reward: [(0, '66.260'), (1, '65.810')] -[2023-10-15 17:35:03,615][52833] Updated weights for policy 0, policy_version 69960 (0.0008) -[2023-10-15 17:35:03,986][52833] Updated weights for policy 0, policy_version 69970 (0.0008) -[2023-10-15 17:35:04,358][52833] Updated weights for policy 0, policy_version 69980 (0.0009) -[2023-10-15 17:35:05,834][52866] Updated weights for policy 1, policy_version 70210 (0.0009) -[2023-10-15 17:35:06,210][52866] Updated weights for policy 1, policy_version 70220 (0.0007) -[2023-10-15 17:35:06,572][52866] Updated weights for policy 1, policy_version 70230 (0.0008) -[2023-10-15 17:35:06,939][52866] Updated weights for policy 1, policy_version 70240 (0.0007) -[2023-10-15 17:35:08,150][52833] Updated weights for policy 0, policy_version 69990 (0.0009) -[2023-10-15 17:35:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 143589376. Throughput: 0: 1801.4, 1: 1800.7. Samples: 35906842. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:35:08,441][51532] Avg episode reward: [(0, '66.560'), (1, '65.620')] -[2023-10-15 17:35:08,517][52833] Updated weights for policy 0, policy_version 70000 (0.0008) -[2023-10-15 17:35:08,889][52833] Updated weights for policy 0, policy_version 70010 (0.0008) -[2023-10-15 17:35:10,554][52866] Updated weights for policy 1, policy_version 70250 (0.0009) -[2023-10-15 17:35:10,917][52866] Updated weights for policy 1, policy_version 70260 (0.0008) -[2023-10-15 17:35:11,272][52866] Updated weights for policy 1, policy_version 70270 (0.0009) -[2023-10-15 17:35:12,578][52833] Updated weights for policy 0, policy_version 70020 (0.0009) -[2023-10-15 17:35:12,948][52833] Updated weights for policy 0, policy_version 70030 (0.0009) -[2023-10-15 17:35:13,319][52833] Updated weights for policy 0, policy_version 70040 (0.0010) -[2023-10-15 17:35:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 143654912. Throughput: 0: 1812.0, 1: 1795.9. Samples: 35928912. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:35:13,442][51532] Avg episode reward: [(0, '67.840'), (1, '63.600')] -[2023-10-15 17:35:15,038][52866] Updated weights for policy 1, policy_version 70280 (0.0010) -[2023-10-15 17:35:15,405][52866] Updated weights for policy 1, policy_version 70290 (0.0008) -[2023-10-15 17:35:15,776][52866] Updated weights for policy 1, policy_version 70300 (0.0008) -[2023-10-15 17:35:16,996][52833] Updated weights for policy 0, policy_version 70050 (0.0010) -[2023-10-15 17:35:17,367][52833] Updated weights for policy 0, policy_version 70060 (0.0010) -[2023-10-15 17:35:17,741][52833] Updated weights for policy 0, policy_version 70070 (0.0010) -[2023-10-15 17:35:18,106][52833] Updated weights for policy 0, policy_version 70080 (0.0008) -[2023-10-15 17:35:18,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 143753216. Throughput: 0: 1807.0, 1: 1791.6. Samples: 35939464. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:35:18,441][51532] Avg episode reward: [(0, '69.710'), (1, '62.260')] -[2023-10-15 17:35:18,442][52410] Saving new best policy, reward=69.710! -[2023-10-15 17:35:19,294][52866] Updated weights for policy 1, policy_version 70310 (0.0009) -[2023-10-15 17:35:19,657][52866] Updated weights for policy 1, policy_version 70320 (0.0008) -[2023-10-15 17:35:20,022][52866] Updated weights for policy 1, policy_version 70330 (0.0009) -[2023-10-15 17:35:21,953][52833] Updated weights for policy 0, policy_version 70090 (0.0008) -[2023-10-15 17:35:22,331][52833] Updated weights for policy 0, policy_version 70100 (0.0009) -[2023-10-15 17:35:22,706][52833] Updated weights for policy 0, policy_version 70110 (0.0008) -[2023-10-15 17:35:23,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 143818752. Throughput: 0: 1814.0, 1: 1792.1. Samples: 35961570. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:35:23,441][51532] Avg episode reward: [(0, '67.260'), (1, '58.530')] -[2023-10-15 17:35:23,761][52866] Updated weights for policy 1, policy_version 70340 (0.0009) -[2023-10-15 17:35:24,125][52866] Updated weights for policy 1, policy_version 70350 (0.0008) -[2023-10-15 17:35:24,496][52866] Updated weights for policy 1, policy_version 70360 (0.0010) -[2023-10-15 17:35:26,436][52833] Updated weights for policy 0, policy_version 70120 (0.0008) -[2023-10-15 17:35:26,806][52833] Updated weights for policy 0, policy_version 70130 (0.0010) -[2023-10-15 17:35:27,174][52833] Updated weights for policy 0, policy_version 70140 (0.0007) -[2023-10-15 17:35:28,321][52866] Updated weights for policy 1, policy_version 70370 (0.0010) -[2023-10-15 17:35:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 143884288. Throughput: 0: 1800.1, 1: 1802.6. Samples: 35982878. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:35:28,442][51532] Avg episode reward: [(0, '66.340'), (1, '56.920')] -[2023-10-15 17:35:28,686][52866] Updated weights for policy 1, policy_version 70380 (0.0007) -[2023-10-15 17:35:29,058][52866] Updated weights for policy 1, policy_version 70390 (0.0009) -[2023-10-15 17:35:29,415][52866] Updated weights for policy 1, policy_version 70400 (0.0007) -[2023-10-15 17:35:31,027][52833] Updated weights for policy 0, policy_version 70150 (0.0009) -[2023-10-15 17:35:31,405][52833] Updated weights for policy 0, policy_version 70160 (0.0010) -[2023-10-15 17:35:31,772][52833] Updated weights for policy 0, policy_version 70170 (0.0011) -[2023-10-15 17:35:33,211][52866] Updated weights for policy 1, policy_version 70410 (0.0008) -[2023-10-15 17:35:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 143949824. Throughput: 0: 1812.1, 1: 1793.4. Samples: 35993992. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:35:33,441][51532] Avg episode reward: [(0, '66.450'), (1, '56.860')] -[2023-10-15 17:35:33,570][52866] Updated weights for policy 1, policy_version 70420 (0.0007) -[2023-10-15 17:35:33,932][52866] Updated weights for policy 1, policy_version 70430 (0.0007) -[2023-10-15 17:35:35,451][52833] Updated weights for policy 0, policy_version 70180 (0.0011) -[2023-10-15 17:35:35,821][52833] Updated weights for policy 0, policy_version 70190 (0.0009) -[2023-10-15 17:35:36,183][52833] Updated weights for policy 0, policy_version 70200 (0.0009) -[2023-10-15 17:35:37,459][52866] Updated weights for policy 1, policy_version 70440 (0.0008) -[2023-10-15 17:35:37,822][52866] Updated weights for policy 1, policy_version 70450 (0.0007) -[2023-10-15 17:35:38,186][52866] Updated weights for policy 1, policy_version 70460 (0.0010) -[2023-10-15 17:35:38,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 144048128. Throughput: 0: 1799.1, 1: 1798.4. Samples: 36015134. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-15 17:35:38,442][51532] Avg episode reward: [(0, '67.400'), (1, '57.910')] -[2023-10-15 17:35:39,943][52833] Updated weights for policy 0, policy_version 70210 (0.0009) -[2023-10-15 17:35:40,318][52833] Updated weights for policy 0, policy_version 70220 (0.0008) -[2023-10-15 17:35:40,691][52833] Updated weights for policy 0, policy_version 70230 (0.0010) -[2023-10-15 17:35:41,056][52833] Updated weights for policy 0, policy_version 70240 (0.0009) -[2023-10-15 17:35:42,158][52866] Updated weights for policy 1, policy_version 70470 (0.0010) -[2023-10-15 17:35:42,522][52866] Updated weights for policy 1, policy_version 70480 (0.0010) -[2023-10-15 17:35:42,884][52866] Updated weights for policy 1, policy_version 70490 (0.0010) -[2023-10-15 17:35:43,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 144113664. Throughput: 0: 1800.2, 1: 1794.4. Samples: 36036122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:35:43,442][51532] Avg episode reward: [(0, '63.150'), (1, '58.430')] -[2023-10-15 17:35:43,456][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000070240_71925760.pth... -[2023-10-15 17:35:43,456][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000070496_72187904.pth... -[2023-10-15 17:35:43,494][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000068576_70221824.pth -[2023-10-15 17:35:43,496][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000068800_70451200.pth -[2023-10-15 17:35:44,814][52833] Updated weights for policy 0, policy_version 70250 (0.0009) -[2023-10-15 17:35:45,191][52833] Updated weights for policy 0, policy_version 70260 (0.0010) -[2023-10-15 17:35:45,562][52833] Updated weights for policy 0, policy_version 70270 (0.0008) -[2023-10-15 17:35:46,535][52866] Updated weights for policy 1, policy_version 70500 (0.0008) -[2023-10-15 17:35:46,909][52866] Updated weights for policy 1, policy_version 70510 (0.0009) -[2023-10-15 17:35:47,264][52866] Updated weights for policy 1, policy_version 70520 (0.0007) -[2023-10-15 17:35:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 144179200. Throughput: 0: 1792.7, 1: 1795.6. Samples: 36047366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:35:48,442][51532] Avg episode reward: [(0, '63.080'), (1, '55.000')] -[2023-10-15 17:35:49,366][52833] Updated weights for policy 0, policy_version 70280 (0.0009) -[2023-10-15 17:35:49,725][52833] Updated weights for policy 0, policy_version 70290 (0.0011) -[2023-10-15 17:35:50,097][52833] Updated weights for policy 0, policy_version 70300 (0.0010) -[2023-10-15 17:35:50,988][52866] Updated weights for policy 1, policy_version 70530 (0.0007) -[2023-10-15 17:35:51,353][52866] Updated weights for policy 1, policy_version 70540 (0.0011) -[2023-10-15 17:35:51,708][52866] Updated weights for policy 1, policy_version 70550 (0.0008) -[2023-10-15 17:35:52,073][52866] Updated weights for policy 1, policy_version 70560 (0.0008) -[2023-10-15 17:35:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144244736. Throughput: 0: 1792.6, 1: 1803.3. Samples: 36068660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:35:53,442][51532] Avg episode reward: [(0, '63.910'), (1, '53.120')] -[2023-10-15 17:35:53,741][52833] Updated weights for policy 0, policy_version 70310 (0.0010) -[2023-10-15 17:35:54,122][52833] Updated weights for policy 0, policy_version 70320 (0.0009) -[2023-10-15 17:35:54,487][52833] Updated weights for policy 0, policy_version 70330 (0.0008) -[2023-10-15 17:35:55,826][52866] Updated weights for policy 1, policy_version 70570 (0.0010) -[2023-10-15 17:35:56,179][52866] Updated weights for policy 1, policy_version 70580 (0.0009) -[2023-10-15 17:35:56,542][52866] Updated weights for policy 1, policy_version 70590 (0.0009) -[2023-10-15 17:35:58,200][52833] Updated weights for policy 0, policy_version 70340 (0.0007) -[2023-10-15 17:35:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144310272. Throughput: 0: 1810.7, 1: 1794.5. Samples: 36091144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:35:58,442][51532] Avg episode reward: [(0, '63.380'), (1, '49.100')] -[2023-10-15 17:35:58,566][52833] Updated weights for policy 0, policy_version 70350 (0.0008) -[2023-10-15 17:35:58,934][52833] Updated weights for policy 0, policy_version 70360 (0.0007) -[2023-10-15 17:36:00,344][52866] Updated weights for policy 1, policy_version 70600 (0.0010) -[2023-10-15 17:36:00,721][52866] Updated weights for policy 1, policy_version 70610 (0.0011) -[2023-10-15 17:36:01,077][52866] Updated weights for policy 1, policy_version 70620 (0.0011) -[2023-10-15 17:36:02,704][52833] Updated weights for policy 0, policy_version 70370 (0.0007) -[2023-10-15 17:36:03,077][52833] Updated weights for policy 0, policy_version 70380 (0.0010) -[2023-10-15 17:36:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 144375808. Throughput: 0: 1790.1, 1: 1807.4. Samples: 36101352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:03,441][51532] Avg episode reward: [(0, '60.210'), (1, '50.930')] -[2023-10-15 17:36:03,448][52833] Updated weights for policy 0, policy_version 70390 (0.0009) -[2023-10-15 17:36:03,821][52833] Updated weights for policy 0, policy_version 70400 (0.0007) -[2023-10-15 17:36:04,719][52866] Updated weights for policy 1, policy_version 70630 (0.0008) -[2023-10-15 17:36:05,094][52866] Updated weights for policy 1, policy_version 70640 (0.0008) -[2023-10-15 17:36:05,468][52866] Updated weights for policy 1, policy_version 70650 (0.0011) -[2023-10-15 17:36:07,645][52833] Updated weights for policy 0, policy_version 70410 (0.0008) -[2023-10-15 17:36:08,015][52833] Updated weights for policy 0, policy_version 70420 (0.0007) -[2023-10-15 17:36:08,388][52833] Updated weights for policy 0, policy_version 70430 (0.0008) -[2023-10-15 17:36:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 144441344. Throughput: 0: 1795.0, 1: 1806.3. Samples: 36123632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:08,442][51532] Avg episode reward: [(0, '57.820'), (1, '53.760')] -[2023-10-15 17:36:09,049][52866] Updated weights for policy 1, policy_version 70660 (0.0009) -[2023-10-15 17:36:09,417][52866] Updated weights for policy 1, policy_version 70670 (0.0007) -[2023-10-15 17:36:09,780][52866] Updated weights for policy 1, policy_version 70680 (0.0007) -[2023-10-15 17:36:12,090][52833] Updated weights for policy 0, policy_version 70440 (0.0009) -[2023-10-15 17:36:12,469][52833] Updated weights for policy 0, policy_version 70450 (0.0008) -[2023-10-15 17:36:12,838][52833] Updated weights for policy 0, policy_version 70460 (0.0008) -[2023-10-15 17:36:13,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 144539648. Throughput: 0: 1789.6, 1: 1808.0. Samples: 36144772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:13,442][51532] Avg episode reward: [(0, '57.800'), (1, '53.950')] -[2023-10-15 17:36:13,649][52866] Updated weights for policy 1, policy_version 70690 (0.0008) -[2023-10-15 17:36:14,006][52866] Updated weights for policy 1, policy_version 70700 (0.0010) -[2023-10-15 17:36:14,377][52866] Updated weights for policy 1, policy_version 70710 (0.0009) -[2023-10-15 17:36:14,738][52866] Updated weights for policy 1, policy_version 70720 (0.0007) -[2023-10-15 17:36:16,636][52833] Updated weights for policy 0, policy_version 70470 (0.0008) -[2023-10-15 17:36:17,014][52833] Updated weights for policy 0, policy_version 70480 (0.0008) -[2023-10-15 17:36:17,382][52833] Updated weights for policy 0, policy_version 70490 (0.0010) -[2023-10-15 17:36:18,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 144605184. Throughput: 0: 1787.5, 1: 1810.2. Samples: 36155888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:18,442][51532] Avg episode reward: [(0, '60.110'), (1, '52.260')] -[2023-10-15 17:36:18,534][52866] Updated weights for policy 1, policy_version 70730 (0.0010) -[2023-10-15 17:36:18,899][52866] Updated weights for policy 1, policy_version 70740 (0.0010) -[2023-10-15 17:36:19,272][52866] Updated weights for policy 1, policy_version 70750 (0.0007) -[2023-10-15 17:36:21,150][52833] Updated weights for policy 0, policy_version 70500 (0.0010) -[2023-10-15 17:36:21,522][52833] Updated weights for policy 0, policy_version 70510 (0.0007) -[2023-10-15 17:36:21,893][52833] Updated weights for policy 0, policy_version 70520 (0.0007) -[2023-10-15 17:36:23,034][52866] Updated weights for policy 1, policy_version 70760 (0.0007) -[2023-10-15 17:36:23,402][52866] Updated weights for policy 1, policy_version 70770 (0.0007) -[2023-10-15 17:36:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 144670720. Throughput: 0: 1792.8, 1: 1811.7. Samples: 36177338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:23,442][51532] Avg episode reward: [(0, '61.840'), (1, '53.400')] -[2023-10-15 17:36:23,776][52866] Updated weights for policy 1, policy_version 70780 (0.0010) -[2023-10-15 17:36:25,617][52833] Updated weights for policy 0, policy_version 70530 (0.0009) -[2023-10-15 17:36:25,996][52833] Updated weights for policy 0, policy_version 70540 (0.0008) -[2023-10-15 17:36:26,362][52833] Updated weights for policy 0, policy_version 70550 (0.0009) -[2023-10-15 17:36:26,724][52833] Updated weights for policy 0, policy_version 70560 (0.0009) -[2023-10-15 17:36:27,478][52866] Updated weights for policy 1, policy_version 70790 (0.0008) -[2023-10-15 17:36:27,856][52866] Updated weights for policy 1, policy_version 70800 (0.0009) -[2023-10-15 17:36:28,224][52866] Updated weights for policy 1, policy_version 70810 (0.0009) -[2023-10-15 17:36:28,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 144769024. Throughput: 0: 1784.0, 1: 1825.9. Samples: 36198566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:28,442][51532] Avg episode reward: [(0, '60.550'), (1, '54.750')] -[2023-10-15 17:36:30,581][52833] Updated weights for policy 0, policy_version 70570 (0.0010) -[2023-10-15 17:36:30,954][52833] Updated weights for policy 0, policy_version 70580 (0.0009) -[2023-10-15 17:36:31,332][52833] Updated weights for policy 0, policy_version 70590 (0.0010) -[2023-10-15 17:36:31,835][52866] Updated weights for policy 1, policy_version 70820 (0.0009) -[2023-10-15 17:36:32,198][52866] Updated weights for policy 1, policy_version 70830 (0.0007) -[2023-10-15 17:36:32,557][52866] Updated weights for policy 1, policy_version 70840 (0.0007) -[2023-10-15 17:36:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 144834560. Throughput: 0: 1800.0, 1: 1808.3. Samples: 36209742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:33,442][51532] Avg episode reward: [(0, '62.200'), (1, '53.980')] -[2023-10-15 17:36:35,116][52833] Updated weights for policy 0, policy_version 70600 (0.0007) -[2023-10-15 17:36:35,484][52833] Updated weights for policy 0, policy_version 70610 (0.0008) -[2023-10-15 17:36:35,856][52833] Updated weights for policy 0, policy_version 70620 (0.0007) -[2023-10-15 17:36:36,301][52866] Updated weights for policy 1, policy_version 70850 (0.0007) -[2023-10-15 17:36:36,670][52866] Updated weights for policy 1, policy_version 70860 (0.0010) -[2023-10-15 17:36:37,030][52866] Updated weights for policy 1, policy_version 70870 (0.0010) -[2023-10-15 17:36:37,400][52866] Updated weights for policy 1, policy_version 70880 (0.0009) -[2023-10-15 17:36:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144900096. Throughput: 0: 1787.8, 1: 1818.1. Samples: 36230924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:38,441][51532] Avg episode reward: [(0, '61.690'), (1, '55.450')] -[2023-10-15 17:36:39,579][52833] Updated weights for policy 0, policy_version 70630 (0.0009) -[2023-10-15 17:36:39,941][52833] Updated weights for policy 0, policy_version 70640 (0.0008) -[2023-10-15 17:36:40,312][52833] Updated weights for policy 0, policy_version 70650 (0.0008) -[2023-10-15 17:36:41,078][52866] Updated weights for policy 1, policy_version 70890 (0.0010) -[2023-10-15 17:36:41,440][52866] Updated weights for policy 1, policy_version 70900 (0.0008) -[2023-10-15 17:36:41,811][52866] Updated weights for policy 1, policy_version 70910 (0.0009) -[2023-10-15 17:36:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144965632. Throughput: 0: 1784.6, 1: 1811.3. Samples: 36252958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:43,441][51532] Avg episode reward: [(0, '58.320'), (1, '57.430')] -[2023-10-15 17:36:43,938][52833] Updated weights for policy 0, policy_version 70660 (0.0007) -[2023-10-15 17:36:44,307][52833] Updated weights for policy 0, policy_version 70670 (0.0007) -[2023-10-15 17:36:44,680][52833] Updated weights for policy 0, policy_version 70680 (0.0008) -[2023-10-15 17:36:45,550][52866] Updated weights for policy 1, policy_version 70920 (0.0011) -[2023-10-15 17:36:45,916][52866] Updated weights for policy 1, policy_version 70930 (0.0010) -[2023-10-15 17:36:46,269][52866] Updated weights for policy 1, policy_version 70940 (0.0007) -[2023-10-15 17:36:48,384][52833] Updated weights for policy 0, policy_version 70690 (0.0009) -[2023-10-15 17:36:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145031168. Throughput: 0: 1786.8, 1: 1814.7. Samples: 36263416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:48,441][51532] Avg episode reward: [(0, '58.010'), (1, '58.180')] -[2023-10-15 17:36:48,751][52833] Updated weights for policy 0, policy_version 70700 (0.0011) -[2023-10-15 17:36:49,127][52833] Updated weights for policy 0, policy_version 70710 (0.0009) -[2023-10-15 17:36:49,497][52833] Updated weights for policy 0, policy_version 70720 (0.0009) -[2023-10-15 17:36:50,241][52866] Updated weights for policy 1, policy_version 70950 (0.0007) -[2023-10-15 17:36:50,607][52866] Updated weights for policy 1, policy_version 70960 (0.0010) -[2023-10-15 17:36:50,972][52866] Updated weights for policy 1, policy_version 70970 (0.0010) -[2023-10-15 17:36:53,374][52833] Updated weights for policy 0, policy_version 70730 (0.0008) -[2023-10-15 17:36:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 145096704. Throughput: 0: 1790.5, 1: 1798.0. Samples: 36285110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:53,442][51532] Avg episode reward: [(0, '60.270'), (1, '61.480')] -[2023-10-15 17:36:53,753][52833] Updated weights for policy 0, policy_version 70740 (0.0008) -[2023-10-15 17:36:54,123][52833] Updated weights for policy 0, policy_version 70750 (0.0009) -[2023-10-15 17:36:54,592][52866] Updated weights for policy 1, policy_version 70980 (0.0010) -[2023-10-15 17:36:54,960][52866] Updated weights for policy 1, policy_version 70990 (0.0007) -[2023-10-15 17:36:55,331][52866] Updated weights for policy 1, policy_version 71000 (0.0007) -[2023-10-15 17:36:57,817][52833] Updated weights for policy 0, policy_version 70760 (0.0010) -[2023-10-15 17:36:58,192][52833] Updated weights for policy 0, policy_version 70770 (0.0008) -[2023-10-15 17:36:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 145162240. Throughput: 0: 1813.8, 1: 1798.0. Samples: 36307300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:36:58,441][51532] Avg episode reward: [(0, '62.080'), (1, '60.140')] -[2023-10-15 17:36:58,554][52833] Updated weights for policy 0, policy_version 70780 (0.0009) -[2023-10-15 17:36:58,960][52866] Updated weights for policy 1, policy_version 71010 (0.0009) -[2023-10-15 17:36:59,324][52866] Updated weights for policy 1, policy_version 71020 (0.0007) -[2023-10-15 17:36:59,698][52866] Updated weights for policy 1, policy_version 71030 (0.0009) -[2023-10-15 17:37:00,057][52866] Updated weights for policy 1, policy_version 71040 (0.0009) -[2023-10-15 17:37:02,345][52833] Updated weights for policy 0, policy_version 70790 (0.0009) -[2023-10-15 17:37:02,726][52833] Updated weights for policy 0, policy_version 70800 (0.0008) -[2023-10-15 17:37:03,099][52833] Updated weights for policy 0, policy_version 70810 (0.0008) -[2023-10-15 17:37:03,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145260544. Throughput: 0: 1796.7, 1: 1798.1. Samples: 36317654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:37:03,441][51532] Avg episode reward: [(0, '61.600'), (1, '58.200')] -[2023-10-15 17:37:03,770][52866] Updated weights for policy 1, policy_version 71050 (0.0009) -[2023-10-15 17:37:04,126][52866] Updated weights for policy 1, policy_version 71060 (0.0008) -[2023-10-15 17:37:04,495][52866] Updated weights for policy 1, policy_version 71070 (0.0007) -[2023-10-15 17:37:06,843][52833] Updated weights for policy 0, policy_version 70820 (0.0007) -[2023-10-15 17:37:07,223][52833] Updated weights for policy 0, policy_version 70830 (0.0008) -[2023-10-15 17:37:07,592][52833] Updated weights for policy 0, policy_version 70840 (0.0009) -[2023-10-15 17:37:08,211][52866] Updated weights for policy 1, policy_version 71080 (0.0010) -[2023-10-15 17:37:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 145326080. Throughput: 0: 1812.3, 1: 1798.9. Samples: 36339840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:37:08,441][51532] Avg episode reward: [(0, '59.950'), (1, '60.070')] -[2023-10-15 17:37:08,578][52866] Updated weights for policy 1, policy_version 71090 (0.0010) -[2023-10-15 17:37:08,945][52866] Updated weights for policy 1, policy_version 71100 (0.0007) -[2023-10-15 17:37:11,254][52833] Updated weights for policy 0, policy_version 70850 (0.0008) -[2023-10-15 17:37:11,628][52833] Updated weights for policy 0, policy_version 70860 (0.0009) -[2023-10-15 17:37:12,006][52833] Updated weights for policy 0, policy_version 70870 (0.0008) -[2023-10-15 17:37:12,376][52833] Updated weights for policy 0, policy_version 70880 (0.0007) -[2023-10-15 17:37:12,770][52866] Updated weights for policy 1, policy_version 71110 (0.0009) -[2023-10-15 17:37:13,142][52866] Updated weights for policy 1, policy_version 71120 (0.0009) -[2023-10-15 17:37:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 145391616. Throughput: 0: 1791.5, 1: 1811.9. Samples: 36360716. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:13,441][51532] Avg episode reward: [(0, '57.520'), (1, '62.380')] -[2023-10-15 17:37:13,511][52866] Updated weights for policy 1, policy_version 71130 (0.0009) -[2023-10-15 17:37:16,085][52833] Updated weights for policy 0, policy_version 70890 (0.0008) -[2023-10-15 17:37:16,453][52833] Updated weights for policy 0, policy_version 70900 (0.0010) -[2023-10-15 17:37:16,831][52833] Updated weights for policy 0, policy_version 70910 (0.0008) -[2023-10-15 17:37:17,253][52866] Updated weights for policy 1, policy_version 71140 (0.0008) -[2023-10-15 17:37:17,621][52866] Updated weights for policy 1, policy_version 71150 (0.0009) -[2023-10-15 17:37:17,986][52866] Updated weights for policy 1, policy_version 71160 (0.0010) -[2023-10-15 17:37:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145489920. Throughput: 0: 1808.4, 1: 1806.2. Samples: 36372398. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:18,442][51532] Avg episode reward: [(0, '57.570'), (1, '64.560')] -[2023-10-15 17:37:20,496][52833] Updated weights for policy 0, policy_version 70920 (0.0007) -[2023-10-15 17:37:20,861][52833] Updated weights for policy 0, policy_version 70930 (0.0007) -[2023-10-15 17:37:21,237][52833] Updated weights for policy 0, policy_version 70940 (0.0009) -[2023-10-15 17:37:21,838][52866] Updated weights for policy 1, policy_version 71170 (0.0010) -[2023-10-15 17:37:22,202][52866] Updated weights for policy 1, policy_version 71180 (0.0008) -[2023-10-15 17:37:22,579][52866] Updated weights for policy 1, policy_version 71190 (0.0007) -[2023-10-15 17:37:22,949][52866] Updated weights for policy 1, policy_version 71200 (0.0008) -[2023-10-15 17:37:23,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145555456. Throughput: 0: 1789.1, 1: 1819.6. Samples: 36393316. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:23,442][51532] Avg episode reward: [(0, '55.410'), (1, '62.340')] -[2023-10-15 17:37:24,906][52833] Updated weights for policy 0, policy_version 70950 (0.0009) -[2023-10-15 17:37:25,279][52833] Updated weights for policy 0, policy_version 70960 (0.0007) -[2023-10-15 17:37:25,661][52833] Updated weights for policy 0, policy_version 70970 (0.0010) -[2023-10-15 17:37:26,599][52866] Updated weights for policy 1, policy_version 71210 (0.0008) -[2023-10-15 17:37:26,964][52866] Updated weights for policy 1, policy_version 71220 (0.0008) -[2023-10-15 17:37:27,335][52866] Updated weights for policy 1, policy_version 71230 (0.0008) -[2023-10-15 17:37:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 145620992. Throughput: 0: 1792.1, 1: 1801.9. Samples: 36414692. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:28,442][51532] Avg episode reward: [(0, '53.920'), (1, '62.010')] -[2023-10-15 17:37:29,423][52833] Updated weights for policy 0, policy_version 70980 (0.0008) -[2023-10-15 17:37:29,798][52833] Updated weights for policy 0, policy_version 70990 (0.0010) -[2023-10-15 17:37:30,163][52833] Updated weights for policy 0, policy_version 71000 (0.0009) -[2023-10-15 17:37:30,913][52866] Updated weights for policy 1, policy_version 71240 (0.0008) -[2023-10-15 17:37:31,276][52866] Updated weights for policy 1, policy_version 71250 (0.0010) -[2023-10-15 17:37:31,643][52866] Updated weights for policy 1, policy_version 71260 (0.0007) -[2023-10-15 17:37:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145686528. Throughput: 0: 1790.0, 1: 1815.5. Samples: 36425664. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:33,442][51532] Avg episode reward: [(0, '53.200'), (1, '64.890')] -[2023-10-15 17:37:34,025][52833] Updated weights for policy 0, policy_version 71010 (0.0010) -[2023-10-15 17:37:34,398][52833] Updated weights for policy 0, policy_version 71020 (0.0009) -[2023-10-15 17:37:34,764][52833] Updated weights for policy 0, policy_version 71030 (0.0008) -[2023-10-15 17:37:35,137][52833] Updated weights for policy 0, policy_version 71040 (0.0008) -[2023-10-15 17:37:35,205][52866] Updated weights for policy 1, policy_version 71270 (0.0008) -[2023-10-15 17:37:35,573][52866] Updated weights for policy 1, policy_version 71280 (0.0009) -[2023-10-15 17:37:35,935][52866] Updated weights for policy 1, policy_version 71290 (0.0009) -[2023-10-15 17:37:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145752064. Throughput: 0: 1789.8, 1: 1809.9. Samples: 36447094. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:38,441][51532] Avg episode reward: [(0, '53.350'), (1, '64.750')] -[2023-10-15 17:37:38,657][52833] Updated weights for policy 0, policy_version 71050 (0.0008) -[2023-10-15 17:37:39,033][52833] Updated weights for policy 0, policy_version 71060 (0.0009) -[2023-10-15 17:37:39,398][52833] Updated weights for policy 0, policy_version 71070 (0.0007) -[2023-10-15 17:37:39,685][52866] Updated weights for policy 1, policy_version 71300 (0.0011) -[2023-10-15 17:37:40,040][52866] Updated weights for policy 1, policy_version 71310 (0.0011) -[2023-10-15 17:37:40,412][52866] Updated weights for policy 1, policy_version 71320 (0.0010) -[2023-10-15 17:37:43,233][52833] Updated weights for policy 0, policy_version 71080 (0.0010) -[2023-10-15 17:37:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 145817600. Throughput: 0: 1804.3, 1: 1810.2. Samples: 36469954. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:43,442][51532] Avg episode reward: [(0, '53.300'), (1, '66.420')] -[2023-10-15 17:37:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000071328_73039872.pth... -[2023-10-15 17:37:43,490][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000069632_71303168.pth -[2023-10-15 17:37:43,602][52833] Updated weights for policy 0, policy_version 71090 (0.0010) -[2023-10-15 17:37:43,971][52833] Updated weights for policy 0, policy_version 71100 (0.0011) -[2023-10-15 17:37:44,115][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000071104_72810496.pth... -[2023-10-15 17:37:44,154][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000069408_71073792.pth -[2023-10-15 17:37:44,169][52866] Updated weights for policy 1, policy_version 71330 (0.0009) -[2023-10-15 17:37:44,529][52866] Updated weights for policy 1, policy_version 71340 (0.0009) -[2023-10-15 17:37:44,893][52866] Updated weights for policy 1, policy_version 71350 (0.0008) -[2023-10-15 17:37:45,255][52866] Updated weights for policy 1, policy_version 71360 (0.0008) -[2023-10-15 17:37:47,828][52833] Updated weights for policy 0, policy_version 71110 (0.0007) -[2023-10-15 17:37:48,213][52833] Updated weights for policy 0, policy_version 71120 (0.0009) -[2023-10-15 17:37:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 145883136. Throughput: 0: 1792.9, 1: 1807.9. Samples: 36479692. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:48,442][51532] Avg episode reward: [(0, '52.110'), (1, '68.890')] -[2023-10-15 17:37:48,575][52833] Updated weights for policy 0, policy_version 71130 (0.0008) -[2023-10-15 17:37:49,096][52866] Updated weights for policy 1, policy_version 71370 (0.0010) -[2023-10-15 17:37:49,466][52866] Updated weights for policy 1, policy_version 71380 (0.0008) -[2023-10-15 17:37:49,834][52866] Updated weights for policy 1, policy_version 71390 (0.0010) -[2023-10-15 17:37:52,342][52833] Updated weights for policy 0, policy_version 71140 (0.0009) -[2023-10-15 17:37:52,722][52833] Updated weights for policy 0, policy_version 71150 (0.0010) -[2023-10-15 17:37:53,089][52833] Updated weights for policy 0, policy_version 71160 (0.0009) -[2023-10-15 17:37:53,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145981440. Throughput: 0: 1798.2, 1: 1801.6. Samples: 36501830. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) -[2023-10-15 17:37:53,442][51532] Avg episode reward: [(0, '50.860'), (1, '72.700')] -[2023-10-15 17:37:53,583][52866] Updated weights for policy 1, policy_version 71400 (0.0010) -[2023-10-15 17:37:53,952][52866] Updated weights for policy 1, policy_version 71410 (0.0009) -[2023-10-15 17:37:54,314][52866] Updated weights for policy 1, policy_version 71420 (0.0009) -[2023-10-15 17:37:56,698][52833] Updated weights for policy 0, policy_version 71170 (0.0009) -[2023-10-15 17:37:57,064][52833] Updated weights for policy 0, policy_version 71180 (0.0010) -[2023-10-15 17:37:57,425][52833] Updated weights for policy 0, policy_version 71190 (0.0009) -[2023-10-15 17:37:57,798][52833] Updated weights for policy 0, policy_version 71200 (0.0008) -[2023-10-15 17:37:57,968][52866] Updated weights for policy 1, policy_version 71430 (0.0008) -[2023-10-15 17:37:58,333][52866] Updated weights for policy 1, policy_version 71440 (0.0009) -[2023-10-15 17:37:58,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 146046976. Throughput: 0: 1791.5, 1: 1813.2. Samples: 36522928. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:37:58,441][51532] Avg episode reward: [(0, '51.360'), (1, '70.570')] -[2023-10-15 17:37:58,697][52866] Updated weights for policy 1, policy_version 71450 (0.0009) -[2023-10-15 17:38:01,521][52833] Updated weights for policy 0, policy_version 71210 (0.0007) -[2023-10-15 17:38:01,884][52833] Updated weights for policy 0, policy_version 71220 (0.0007) -[2023-10-15 17:38:02,253][52833] Updated weights for policy 0, policy_version 71230 (0.0007) -[2023-10-15 17:38:02,448][52866] Updated weights for policy 1, policy_version 71460 (0.0009) -[2023-10-15 17:38:02,816][52866] Updated weights for policy 1, policy_version 71470 (0.0010) -[2023-10-15 17:38:03,186][52866] Updated weights for policy 1, policy_version 71480 (0.0009) -[2023-10-15 17:38:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 146112512. Throughput: 0: 1797.3, 1: 1806.4. Samples: 36534564. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:38:03,441][51532] Avg episode reward: [(0, '51.020'), (1, '72.320')] -[2023-10-15 17:38:05,925][52833] Updated weights for policy 0, policy_version 71240 (0.0009) -[2023-10-15 17:38:06,285][52833] Updated weights for policy 0, policy_version 71250 (0.0009) -[2023-10-15 17:38:06,651][52833] Updated weights for policy 0, policy_version 71260 (0.0007) -[2023-10-15 17:38:06,949][52866] Updated weights for policy 1, policy_version 71490 (0.0011) -[2023-10-15 17:38:07,321][52866] Updated weights for policy 1, policy_version 71500 (0.0008) -[2023-10-15 17:38:07,688][52866] Updated weights for policy 1, policy_version 71510 (0.0007) -[2023-10-15 17:38:08,052][52866] Updated weights for policy 1, policy_version 71520 (0.0007) -[2023-10-15 17:38:08,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 146210816. Throughput: 0: 1795.6, 1: 1810.4. Samples: 36555586. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:38:08,442][51532] Avg episode reward: [(0, '52.110'), (1, '75.050')] -[2023-10-15 17:38:10,332][52833] Updated weights for policy 0, policy_version 71270 (0.0007) -[2023-10-15 17:38:10,695][52833] Updated weights for policy 0, policy_version 71280 (0.0008) -[2023-10-15 17:38:11,067][52833] Updated weights for policy 0, policy_version 71290 (0.0007) -[2023-10-15 17:38:11,792][52866] Updated weights for policy 1, policy_version 71530 (0.0010) -[2023-10-15 17:38:12,159][52866] Updated weights for policy 1, policy_version 71540 (0.0011) -[2023-10-15 17:38:12,518][52866] Updated weights for policy 1, policy_version 71550 (0.0007) -[2023-10-15 17:38:13,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 146276352. Throughput: 0: 1801.3, 1: 1809.5. Samples: 36577178. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:38:13,442][51532] Avg episode reward: [(0, '55.690'), (1, '72.740')] -[2023-10-15 17:38:14,697][52833] Updated weights for policy 0, policy_version 71300 (0.0008) -[2023-10-15 17:38:15,074][52833] Updated weights for policy 0, policy_version 71310 (0.0009) -[2023-10-15 17:38:15,439][52833] Updated weights for policy 0, policy_version 71320 (0.0007) -[2023-10-15 17:38:16,223][52866] Updated weights for policy 1, policy_version 71560 (0.0010) -[2023-10-15 17:38:16,585][52866] Updated weights for policy 1, policy_version 71570 (0.0010) -[2023-10-15 17:38:16,954][52866] Updated weights for policy 1, policy_version 71580 (0.0008) -[2023-10-15 17:38:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 146341888. Throughput: 0: 1802.7, 1: 1813.3. Samples: 36588384. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:38:18,442][51532] Avg episode reward: [(0, '56.300'), (1, '71.140')] -[2023-10-15 17:38:19,050][52833] Updated weights for policy 0, policy_version 71330 (0.0007) -[2023-10-15 17:38:19,423][52833] Updated weights for policy 0, policy_version 71340 (0.0007) -[2023-10-15 17:38:19,795][52833] Updated weights for policy 0, policy_version 71350 (0.0007) -[2023-10-15 17:38:20,162][52833] Updated weights for policy 0, policy_version 71360 (0.0011) -[2023-10-15 17:38:20,735][52866] Updated weights for policy 1, policy_version 71590 (0.0007) -[2023-10-15 17:38:21,104][52866] Updated weights for policy 1, policy_version 71600 (0.0008) -[2023-10-15 17:38:21,468][52866] Updated weights for policy 1, policy_version 71610 (0.0008) -[2023-10-15 17:38:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146407424. Throughput: 0: 1808.8, 1: 1798.8. Samples: 36609434. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:38:23,441][51532] Avg episode reward: [(0, '56.470'), (1, '69.930')] -[2023-10-15 17:38:23,919][52833] Updated weights for policy 0, policy_version 71370 (0.0009) -[2023-10-15 17:38:24,273][52833] Updated weights for policy 0, policy_version 71380 (0.0009) -[2023-10-15 17:38:24,640][52833] Updated weights for policy 0, policy_version 71390 (0.0008) -[2023-10-15 17:38:25,237][52866] Updated weights for policy 1, policy_version 71620 (0.0008) -[2023-10-15 17:38:25,598][52866] Updated weights for policy 1, policy_version 71630 (0.0008) -[2023-10-15 17:38:25,973][52866] Updated weights for policy 1, policy_version 71640 (0.0008) -[2023-10-15 17:38:28,375][52833] Updated weights for policy 0, policy_version 71400 (0.0008) -[2023-10-15 17:38:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146472960. Throughput: 0: 1805.3, 1: 1794.1. Samples: 36631924. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:38:28,441][51532] Avg episode reward: [(0, '57.410'), (1, '68.900')] -[2023-10-15 17:38:28,749][52833] Updated weights for policy 0, policy_version 71410 (0.0009) -[2023-10-15 17:38:29,120][52833] Updated weights for policy 0, policy_version 71420 (0.0009) -[2023-10-15 17:38:29,893][52866] Updated weights for policy 1, policy_version 71650 (0.0009) -[2023-10-15 17:38:30,259][52866] Updated weights for policy 1, policy_version 71660 (0.0010) -[2023-10-15 17:38:30,625][52866] Updated weights for policy 1, policy_version 71670 (0.0008) -[2023-10-15 17:38:30,991][52866] Updated weights for policy 1, policy_version 71680 (0.0007) -[2023-10-15 17:38:32,948][52833] Updated weights for policy 0, policy_version 71430 (0.0007) -[2023-10-15 17:38:33,334][52833] Updated weights for policy 0, policy_version 71440 (0.0007) -[2023-10-15 17:38:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 146538496. Throughput: 0: 1809.8, 1: 1798.0. Samples: 36642042. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:38:33,442][51532] Avg episode reward: [(0, '59.420'), (1, '68.660')] -[2023-10-15 17:38:33,702][52833] Updated weights for policy 0, policy_version 71450 (0.0007) -[2023-10-15 17:38:34,457][52866] Updated weights for policy 1, policy_version 71690 (0.0011) -[2023-10-15 17:38:34,822][52866] Updated weights for policy 1, policy_version 71700 (0.0009) -[2023-10-15 17:38:35,187][52866] Updated weights for policy 1, policy_version 71710 (0.0010) -[2023-10-15 17:38:37,458][52833] Updated weights for policy 0, policy_version 71460 (0.0008) -[2023-10-15 17:38:37,826][52833] Updated weights for policy 0, policy_version 71470 (0.0010) -[2023-10-15 17:38:38,193][52833] Updated weights for policy 0, policy_version 71480 (0.0007) -[2023-10-15 17:38:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 146604032. Throughput: 0: 1810.8, 1: 1806.2. Samples: 36664598. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) -[2023-10-15 17:38:38,441][51532] Avg episode reward: [(0, '56.910'), (1, '68.670')] -[2023-10-15 17:38:38,927][52866] Updated weights for policy 1, policy_version 71720 (0.0007) -[2023-10-15 17:38:39,287][52866] Updated weights for policy 1, policy_version 71730 (0.0007) -[2023-10-15 17:38:39,656][52866] Updated weights for policy 1, policy_version 71740 (0.0008) -[2023-10-15 17:38:41,858][52833] Updated weights for policy 0, policy_version 71490 (0.0008) -[2023-10-15 17:38:42,227][52833] Updated weights for policy 0, policy_version 71500 (0.0007) -[2023-10-15 17:38:42,590][52833] Updated weights for policy 0, policy_version 71510 (0.0008) -[2023-10-15 17:38:42,954][52833] Updated weights for policy 0, policy_version 71520 (0.0008) -[2023-10-15 17:38:43,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 146702336. Throughput: 0: 1817.0, 1: 1805.4. Samples: 36685936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:38:43,441][51532] Avg episode reward: [(0, '57.240'), (1, '67.790')] -[2023-10-15 17:38:43,507][52866] Updated weights for policy 1, policy_version 71750 (0.0008) -[2023-10-15 17:38:43,872][52866] Updated weights for policy 1, policy_version 71760 (0.0010) -[2023-10-15 17:38:44,237][52866] Updated weights for policy 1, policy_version 71770 (0.0010) -[2023-10-15 17:38:46,958][52833] Updated weights for policy 0, policy_version 71530 (0.0008) -[2023-10-15 17:38:47,336][52833] Updated weights for policy 0, policy_version 71540 (0.0007) -[2023-10-15 17:38:47,705][52833] Updated weights for policy 0, policy_version 71550 (0.0007) -[2023-10-15 17:38:48,020][52866] Updated weights for policy 1, policy_version 71780 (0.0009) -[2023-10-15 17:38:48,389][52866] Updated weights for policy 1, policy_version 71790 (0.0010) -[2023-10-15 17:38:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 146767872. Throughput: 0: 1808.2, 1: 1795.3. Samples: 36696722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:38:48,442][51532] Avg episode reward: [(0, '55.050'), (1, '65.580')] -[2023-10-15 17:38:48,750][52866] Updated weights for policy 1, policy_version 71800 (0.0009) -[2023-10-15 17:38:51,445][52833] Updated weights for policy 0, policy_version 71560 (0.0008) -[2023-10-15 17:38:51,819][52833] Updated weights for policy 0, policy_version 71570 (0.0008) -[2023-10-15 17:38:52,185][52833] Updated weights for policy 0, policy_version 71580 (0.0008) -[2023-10-15 17:38:52,558][52866] Updated weights for policy 1, policy_version 71810 (0.0009) -[2023-10-15 17:38:52,934][52866] Updated weights for policy 1, policy_version 71820 (0.0011) -[2023-10-15 17:38:53,292][52866] Updated weights for policy 1, policy_version 71830 (0.0008) -[2023-10-15 17:38:53,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 146833408. Throughput: 0: 1819.4, 1: 1796.7. Samples: 36718310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:38:53,442][51532] Avg episode reward: [(0, '57.390'), (1, '63.980')] -[2023-10-15 17:38:53,657][52866] Updated weights for policy 1, policy_version 71840 (0.0009) -[2023-10-15 17:38:55,961][52833] Updated weights for policy 0, policy_version 71590 (0.0007) -[2023-10-15 17:38:56,336][52833] Updated weights for policy 0, policy_version 71600 (0.0008) -[2023-10-15 17:38:56,699][52833] Updated weights for policy 0, policy_version 71610 (0.0008) -[2023-10-15 17:38:57,434][52866] Updated weights for policy 1, policy_version 71850 (0.0007) -[2023-10-15 17:38:57,788][52866] Updated weights for policy 1, policy_version 71860 (0.0009) -[2023-10-15 17:38:58,163][52866] Updated weights for policy 1, policy_version 71870 (0.0009) -[2023-10-15 17:38:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 146931712. Throughput: 0: 1797.9, 1: 1805.2. Samples: 36739314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:38:58,442][51532] Avg episode reward: [(0, '60.020'), (1, '62.910')] -[2023-10-15 17:39:00,311][52833] Updated weights for policy 0, policy_version 71620 (0.0009) -[2023-10-15 17:39:00,686][52833] Updated weights for policy 0, policy_version 71630 (0.0008) -[2023-10-15 17:39:01,058][52833] Updated weights for policy 0, policy_version 71640 (0.0008) -[2023-10-15 17:39:02,025][52866] Updated weights for policy 1, policy_version 71880 (0.0009) -[2023-10-15 17:39:02,386][52866] Updated weights for policy 1, policy_version 71890 (0.0008) -[2023-10-15 17:39:02,750][52866] Updated weights for policy 1, policy_version 71900 (0.0007) -[2023-10-15 17:39:03,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 146997248. Throughput: 0: 1812.0, 1: 1795.0. Samples: 36750700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:39:03,442][51532] Avg episode reward: [(0, '59.800'), (1, '60.740')] -[2023-10-15 17:39:04,664][52833] Updated weights for policy 0, policy_version 71650 (0.0009) -[2023-10-15 17:39:05,040][52833] Updated weights for policy 0, policy_version 71660 (0.0008) -[2023-10-15 17:39:05,416][52833] Updated weights for policy 0, policy_version 71670 (0.0008) -[2023-10-15 17:39:05,781][52833] Updated weights for policy 0, policy_version 71680 (0.0007) -[2023-10-15 17:39:06,432][52866] Updated weights for policy 1, policy_version 71910 (0.0010) -[2023-10-15 17:39:06,800][52866] Updated weights for policy 1, policy_version 71920 (0.0009) -[2023-10-15 17:39:07,172][52866] Updated weights for policy 1, policy_version 71930 (0.0010) -[2023-10-15 17:39:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147062784. Throughput: 0: 1801.1, 1: 1810.6. Samples: 36771962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:39:08,442][51532] Avg episode reward: [(0, '58.860'), (1, '59.740')] -[2023-10-15 17:39:09,600][52833] Updated weights for policy 0, policy_version 71690 (0.0009) -[2023-10-15 17:39:09,967][52833] Updated weights for policy 0, policy_version 71700 (0.0009) -[2023-10-15 17:39:10,337][52833] Updated weights for policy 0, policy_version 71710 (0.0008) -[2023-10-15 17:39:10,902][52866] Updated weights for policy 1, policy_version 71940 (0.0007) -[2023-10-15 17:39:11,268][52866] Updated weights for policy 1, policy_version 71950 (0.0009) -[2023-10-15 17:39:11,628][52866] Updated weights for policy 1, policy_version 71960 (0.0007) -[2023-10-15 17:39:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 147128320. Throughput: 0: 1800.9, 1: 1805.9. Samples: 36794232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:39:13,442][51532] Avg episode reward: [(0, '59.610'), (1, '59.000')] -[2023-10-15 17:39:14,073][52833] Updated weights for policy 0, policy_version 71720 (0.0008) -[2023-10-15 17:39:14,444][52833] Updated weights for policy 0, policy_version 71730 (0.0008) -[2023-10-15 17:39:14,815][52833] Updated weights for policy 0, policy_version 71740 (0.0007) -[2023-10-15 17:39:15,293][52866] Updated weights for policy 1, policy_version 71970 (0.0007) -[2023-10-15 17:39:15,664][52866] Updated weights for policy 1, policy_version 71980 (0.0010) -[2023-10-15 17:39:16,025][52866] Updated weights for policy 1, policy_version 71990 (0.0008) -[2023-10-15 17:39:16,389][52866] Updated weights for policy 1, policy_version 72000 (0.0009) -[2023-10-15 17:39:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147193856. Throughput: 0: 1800.6, 1: 1814.1. Samples: 36804704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:39:18,442][51532] Avg episode reward: [(0, '60.600'), (1, '59.800')] -[2023-10-15 17:39:18,477][52833] Updated weights for policy 0, policy_version 71750 (0.0009) -[2023-10-15 17:39:18,858][52833] Updated weights for policy 0, policy_version 71760 (0.0007) -[2023-10-15 17:39:19,230][52833] Updated weights for policy 0, policy_version 71770 (0.0007) -[2023-10-15 17:39:19,991][52866] Updated weights for policy 1, policy_version 72010 (0.0011) -[2023-10-15 17:39:20,372][52866] Updated weights for policy 1, policy_version 72020 (0.0010) -[2023-10-15 17:39:20,739][52866] Updated weights for policy 1, policy_version 72030 (0.0007) -[2023-10-15 17:39:22,742][52833] Updated weights for policy 0, policy_version 71780 (0.0009) -[2023-10-15 17:39:23,106][52833] Updated weights for policy 0, policy_version 71790 (0.0010) -[2023-10-15 17:39:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 147259392. Throughput: 0: 1804.2, 1: 1795.5. Samples: 36826588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:39:23,442][51532] Avg episode reward: [(0, '59.400'), (1, '60.260')] -[2023-10-15 17:39:23,467][52833] Updated weights for policy 0, policy_version 71800 (0.0010) -[2023-10-15 17:39:24,486][52866] Updated weights for policy 1, policy_version 72040 (0.0008) -[2023-10-15 17:39:24,855][52866] Updated weights for policy 1, policy_version 72050 (0.0008) -[2023-10-15 17:39:25,224][52866] Updated weights for policy 1, policy_version 72060 (0.0010) -[2023-10-15 17:39:27,211][52833] Updated weights for policy 0, policy_version 71810 (0.0010) -[2023-10-15 17:39:27,584][52833] Updated weights for policy 0, policy_version 71820 (0.0008) -[2023-10-15 17:39:27,953][52833] Updated weights for policy 0, policy_version 71830 (0.0007) -[2023-10-15 17:39:28,319][52833] Updated weights for policy 0, policy_version 71840 (0.0007) -[2023-10-15 17:39:28,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 147357696. Throughput: 0: 1819.3, 1: 1793.5. Samples: 36848512. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:39:28,442][51532] Avg episode reward: [(0, '59.170'), (1, '60.970')] -[2023-10-15 17:39:29,133][52866] Updated weights for policy 1, policy_version 72070 (0.0009) -[2023-10-15 17:39:29,518][52866] Updated weights for policy 1, policy_version 72080 (0.0008) -[2023-10-15 17:39:29,876][52866] Updated weights for policy 1, policy_version 72090 (0.0010) -[2023-10-15 17:39:31,863][52833] Updated weights for policy 0, policy_version 71850 (0.0007) -[2023-10-15 17:39:32,244][52833] Updated weights for policy 0, policy_version 71860 (0.0007) -[2023-10-15 17:39:32,623][52833] Updated weights for policy 0, policy_version 71870 (0.0008) -[2023-10-15 17:39:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 147423232. Throughput: 0: 1816.7, 1: 1794.3. Samples: 36859222. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:39:33,442][51532] Avg episode reward: [(0, '64.620'), (1, '59.300')] -[2023-10-15 17:39:33,659][52866] Updated weights for policy 1, policy_version 72100 (0.0009) -[2023-10-15 17:39:34,027][52866] Updated weights for policy 1, policy_version 72110 (0.0007) -[2023-10-15 17:39:34,391][52866] Updated weights for policy 1, policy_version 72120 (0.0009) -[2023-10-15 17:39:36,375][52833] Updated weights for policy 0, policy_version 71880 (0.0008) -[2023-10-15 17:39:36,749][52833] Updated weights for policy 0, policy_version 71890 (0.0007) -[2023-10-15 17:39:37,113][52833] Updated weights for policy 0, policy_version 71900 (0.0007) -[2023-10-15 17:39:38,052][52866] Updated weights for policy 1, policy_version 72130 (0.0008) -[2023-10-15 17:39:38,426][52866] Updated weights for policy 1, policy_version 72140 (0.0009) -[2023-10-15 17:39:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 147488768. Throughput: 0: 1815.6, 1: 1793.7. Samples: 36880730. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:39:38,441][51532] Avg episode reward: [(0, '64.320'), (1, '56.870')] -[2023-10-15 17:39:38,796][52866] Updated weights for policy 1, policy_version 72150 (0.0008) -[2023-10-15 17:39:39,161][52866] Updated weights for policy 1, policy_version 72160 (0.0009) -[2023-10-15 17:39:40,740][52833] Updated weights for policy 0, policy_version 71910 (0.0007) -[2023-10-15 17:39:41,106][52833] Updated weights for policy 0, policy_version 71920 (0.0007) -[2023-10-15 17:39:41,474][52833] Updated weights for policy 0, policy_version 71930 (0.0008) -[2023-10-15 17:39:42,743][52866] Updated weights for policy 1, policy_version 72170 (0.0007) -[2023-10-15 17:39:43,110][52866] Updated weights for policy 1, policy_version 72180 (0.0007) -[2023-10-15 17:39:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 147554304. Throughput: 0: 1819.9, 1: 1810.3. Samples: 36902672. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:39:43,441][51532] Avg episode reward: [(0, '65.790'), (1, '58.570')] -[2023-10-15 17:39:43,449][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000071936_73662464.pth... -[2023-10-15 17:39:43,470][52866] Updated weights for policy 1, policy_version 72190 (0.0007) -[2023-10-15 17:39:43,484][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000070240_71925760.pth -[2023-10-15 17:39:43,533][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000072192_73924608.pth... -[2023-10-15 17:39:43,570][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000070496_72187904.pth -[2023-10-15 17:39:45,161][52833] Updated weights for policy 0, policy_version 71940 (0.0009) -[2023-10-15 17:39:45,520][52833] Updated weights for policy 0, policy_version 71950 (0.0007) -[2023-10-15 17:39:45,887][52833] Updated weights for policy 0, policy_version 71960 (0.0008) -[2023-10-15 17:39:47,294][52866] Updated weights for policy 1, policy_version 72200 (0.0008) -[2023-10-15 17:39:47,657][52866] Updated weights for policy 1, policy_version 72210 (0.0009) -[2023-10-15 17:39:48,020][52866] Updated weights for policy 1, policy_version 72220 (0.0008) -[2023-10-15 17:39:48,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 147652608. Throughput: 0: 1816.8, 1: 1804.4. Samples: 36913654. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:39:48,441][51532] Avg episode reward: [(0, '66.880'), (1, '61.640')] -[2023-10-15 17:39:49,814][52833] Updated weights for policy 0, policy_version 71970 (0.0010) -[2023-10-15 17:39:50,180][52833] Updated weights for policy 0, policy_version 71980 (0.0008) -[2023-10-15 17:39:50,539][52833] Updated weights for policy 0, policy_version 71990 (0.0007) -[2023-10-15 17:39:50,911][52833] Updated weights for policy 0, policy_version 72000 (0.0007) -[2023-10-15 17:39:51,795][52866] Updated weights for policy 1, policy_version 72230 (0.0008) -[2023-10-15 17:39:52,157][52866] Updated weights for policy 1, policy_version 72240 (0.0010) -[2023-10-15 17:39:52,536][52866] Updated weights for policy 1, policy_version 72250 (0.0007) -[2023-10-15 17:39:53,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 147718144. Throughput: 0: 1812.8, 1: 1813.0. Samples: 36935120. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:39:53,441][51532] Avg episode reward: [(0, '64.830'), (1, '61.110')] -[2023-10-15 17:39:54,644][52833] Updated weights for policy 0, policy_version 72010 (0.0008) -[2023-10-15 17:39:55,010][52833] Updated weights for policy 0, policy_version 72020 (0.0008) -[2023-10-15 17:39:55,377][52833] Updated weights for policy 0, policy_version 72030 (0.0008) -[2023-10-15 17:39:56,183][52866] Updated weights for policy 1, policy_version 72260 (0.0009) -[2023-10-15 17:39:56,547][52866] Updated weights for policy 1, policy_version 72270 (0.0009) -[2023-10-15 17:39:56,903][52866] Updated weights for policy 1, policy_version 72280 (0.0007) -[2023-10-15 17:39:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 147783680. Throughput: 0: 1811.6, 1: 1800.1. Samples: 36956756. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:39:58,442][51532] Avg episode reward: [(0, '62.810'), (1, '61.770')] -[2023-10-15 17:39:59,150][52833] Updated weights for policy 0, policy_version 72040 (0.0007) -[2023-10-15 17:39:59,524][52833] Updated weights for policy 0, policy_version 72050 (0.0008) -[2023-10-15 17:39:59,881][52833] Updated weights for policy 0, policy_version 72060 (0.0009) -[2023-10-15 17:40:00,612][52866] Updated weights for policy 1, policy_version 72290 (0.0007) -[2023-10-15 17:40:00,972][52866] Updated weights for policy 1, policy_version 72300 (0.0009) -[2023-10-15 17:40:01,342][52866] Updated weights for policy 1, policy_version 72310 (0.0010) -[2023-10-15 17:40:01,708][52866] Updated weights for policy 1, policy_version 72320 (0.0008) -[2023-10-15 17:40:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147849216. Throughput: 0: 1811.5, 1: 1809.3. Samples: 36967636. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:40:03,441][51532] Avg episode reward: [(0, '64.540'), (1, '63.550')] -[2023-10-15 17:40:03,624][52833] Updated weights for policy 0, policy_version 72070 (0.0009) -[2023-10-15 17:40:04,005][52833] Updated weights for policy 0, policy_version 72080 (0.0008) -[2023-10-15 17:40:04,375][52833] Updated weights for policy 0, policy_version 72090 (0.0009) -[2023-10-15 17:40:05,350][52866] Updated weights for policy 1, policy_version 72330 (0.0008) -[2023-10-15 17:40:05,713][52866] Updated weights for policy 1, policy_version 72340 (0.0009) -[2023-10-15 17:40:06,089][52866] Updated weights for policy 1, policy_version 72350 (0.0008) -[2023-10-15 17:40:08,068][52833] Updated weights for policy 0, policy_version 72100 (0.0009) -[2023-10-15 17:40:08,436][52833] Updated weights for policy 0, policy_version 72110 (0.0009) -[2023-10-15 17:40:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 147914752. Throughput: 0: 1810.0, 1: 1804.7. Samples: 36989248. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:40:08,442][51532] Avg episode reward: [(0, '65.240'), (1, '61.500')] -[2023-10-15 17:40:08,810][52833] Updated weights for policy 0, policy_version 72120 (0.0007) -[2023-10-15 17:40:09,640][52866] Updated weights for policy 1, policy_version 72360 (0.0009) -[2023-10-15 17:40:10,005][52866] Updated weights for policy 1, policy_version 72370 (0.0010) -[2023-10-15 17:40:10,365][52866] Updated weights for policy 1, policy_version 72380 (0.0009) -[2023-10-15 17:40:12,814][52833] Updated weights for policy 0, policy_version 72130 (0.0009) -[2023-10-15 17:40:13,177][52833] Updated weights for policy 0, policy_version 72140 (0.0010) -[2023-10-15 17:40:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 147980288. Throughput: 0: 1814.2, 1: 1809.0. Samples: 37011554. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-15 17:40:13,442][51532] Avg episode reward: [(0, '63.640'), (1, '63.130')] -[2023-10-15 17:40:13,548][52833] Updated weights for policy 0, policy_version 72150 (0.0011) -[2023-10-15 17:40:13,921][52833] Updated weights for policy 0, policy_version 72160 (0.0009) -[2023-10-15 17:40:14,202][52866] Updated weights for policy 1, policy_version 72390 (0.0011) -[2023-10-15 17:40:14,581][52866] Updated weights for policy 1, policy_version 72400 (0.0011) -[2023-10-15 17:40:14,952][52866] Updated weights for policy 1, policy_version 72410 (0.0007) -[2023-10-15 17:40:17,652][52833] Updated weights for policy 0, policy_version 72170 (0.0010) -[2023-10-15 17:40:18,027][52833] Updated weights for policy 0, policy_version 72180 (0.0009) -[2023-10-15 17:40:18,394][52833] Updated weights for policy 0, policy_version 72190 (0.0007) -[2023-10-15 17:40:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 148045824. Throughput: 0: 1793.5, 1: 1810.0. Samples: 37021378. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:18,442][51532] Avg episode reward: [(0, '62.700'), (1, '63.590')] -[2023-10-15 17:40:18,603][52866] Updated weights for policy 1, policy_version 72420 (0.0007) -[2023-10-15 17:40:18,977][52866] Updated weights for policy 1, policy_version 72430 (0.0008) -[2023-10-15 17:40:19,343][52866] Updated weights for policy 1, policy_version 72440 (0.0009) -[2023-10-15 17:40:22,001][52833] Updated weights for policy 0, policy_version 72200 (0.0007) -[2023-10-15 17:40:22,356][52833] Updated weights for policy 0, policy_version 72210 (0.0008) -[2023-10-15 17:40:22,722][52833] Updated weights for policy 0, policy_version 72220 (0.0008) -[2023-10-15 17:40:23,121][52866] Updated weights for policy 1, policy_version 72450 (0.0008) -[2023-10-15 17:40:23,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 148144128. Throughput: 0: 1814.3, 1: 1808.0. Samples: 37043734. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:23,442][51532] Avg episode reward: [(0, '57.500'), (1, '63.070')] -[2023-10-15 17:40:23,491][52866] Updated weights for policy 1, policy_version 72460 (0.0007) -[2023-10-15 17:40:23,858][52866] Updated weights for policy 1, policy_version 72470 (0.0010) -[2023-10-15 17:40:24,225][52866] Updated weights for policy 1, policy_version 72480 (0.0007) -[2023-10-15 17:40:26,479][52833] Updated weights for policy 0, policy_version 72230 (0.0008) -[2023-10-15 17:40:26,847][52833] Updated weights for policy 0, policy_version 72240 (0.0009) -[2023-10-15 17:40:27,223][52833] Updated weights for policy 0, policy_version 72250 (0.0009) -[2023-10-15 17:40:28,117][52866] Updated weights for policy 1, policy_version 72490 (0.0008) -[2023-10-15 17:40:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 148209664. Throughput: 0: 1790.7, 1: 1810.0. Samples: 37064704. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:28,442][51532] Avg episode reward: [(0, '59.890'), (1, '66.080')] -[2023-10-15 17:40:28,496][52866] Updated weights for policy 1, policy_version 72500 (0.0008) -[2023-10-15 17:40:28,852][52866] Updated weights for policy 1, policy_version 72510 (0.0007) -[2023-10-15 17:40:30,915][52833] Updated weights for policy 0, policy_version 72260 (0.0011) -[2023-10-15 17:40:31,287][52833] Updated weights for policy 0, policy_version 72270 (0.0008) -[2023-10-15 17:40:31,654][52833] Updated weights for policy 0, policy_version 72280 (0.0008) -[2023-10-15 17:40:32,621][52866] Updated weights for policy 1, policy_version 72520 (0.0008) -[2023-10-15 17:40:32,986][52866] Updated weights for policy 1, policy_version 72530 (0.0009) -[2023-10-15 17:40:33,340][52866] Updated weights for policy 1, policy_version 72540 (0.0009) -[2023-10-15 17:40:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 148275200. Throughput: 0: 1811.3, 1: 1800.1. Samples: 37076168. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:33,442][51532] Avg episode reward: [(0, '60.050'), (1, '67.250')] -[2023-10-15 17:40:35,273][52833] Updated weights for policy 0, policy_version 72290 (0.0008) -[2023-10-15 17:40:35,645][52833] Updated weights for policy 0, policy_version 72300 (0.0008) -[2023-10-15 17:40:36,013][52833] Updated weights for policy 0, policy_version 72310 (0.0009) -[2023-10-15 17:40:36,377][52833] Updated weights for policy 0, policy_version 72320 (0.0008) -[2023-10-15 17:40:37,177][52866] Updated weights for policy 1, policy_version 72550 (0.0011) -[2023-10-15 17:40:37,538][52866] Updated weights for policy 1, policy_version 72560 (0.0009) -[2023-10-15 17:40:37,908][52866] Updated weights for policy 1, policy_version 72570 (0.0009) -[2023-10-15 17:40:38,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 148373504. Throughput: 0: 1792.7, 1: 1811.6. Samples: 37097312. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:38,442][51532] Avg episode reward: [(0, '58.350'), (1, '65.950')] -[2023-10-15 17:40:40,054][52833] Updated weights for policy 0, policy_version 72330 (0.0008) -[2023-10-15 17:40:40,426][52833] Updated weights for policy 0, policy_version 72340 (0.0009) -[2023-10-15 17:40:40,793][52833] Updated weights for policy 0, policy_version 72350 (0.0008) -[2023-10-15 17:40:41,794][52866] Updated weights for policy 1, policy_version 72580 (0.0009) -[2023-10-15 17:40:42,155][52866] Updated weights for policy 1, policy_version 72590 (0.0010) -[2023-10-15 17:40:42,520][52866] Updated weights for policy 1, policy_version 72600 (0.0011) -[2023-10-15 17:40:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 148439040. Throughput: 0: 1792.7, 1: 1798.4. Samples: 37118356. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:43,442][51532] Avg episode reward: [(0, '58.130'), (1, '63.370')] -[2023-10-15 17:40:44,703][52833] Updated weights for policy 0, policy_version 72360 (0.0007) -[2023-10-15 17:40:45,071][52833] Updated weights for policy 0, policy_version 72370 (0.0010) -[2023-10-15 17:40:45,434][52833] Updated weights for policy 0, policy_version 72380 (0.0010) -[2023-10-15 17:40:46,177][52866] Updated weights for policy 1, policy_version 72610 (0.0008) -[2023-10-15 17:40:46,541][52866] Updated weights for policy 1, policy_version 72620 (0.0007) -[2023-10-15 17:40:46,910][52866] Updated weights for policy 1, policy_version 72630 (0.0009) -[2023-10-15 17:40:47,275][52866] Updated weights for policy 1, policy_version 72640 (0.0008) -[2023-10-15 17:40:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148504576. Throughput: 0: 1788.3, 1: 1813.4. Samples: 37129712. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:48,441][51532] Avg episode reward: [(0, '58.860'), (1, '62.210')] -[2023-10-15 17:40:49,255][52833] Updated weights for policy 0, policy_version 72390 (0.0009) -[2023-10-15 17:40:49,628][52833] Updated weights for policy 0, policy_version 72400 (0.0010) -[2023-10-15 17:40:50,007][52833] Updated weights for policy 0, policy_version 72410 (0.0007) -[2023-10-15 17:40:51,134][52866] Updated weights for policy 1, policy_version 72650 (0.0009) -[2023-10-15 17:40:51,505][52866] Updated weights for policy 1, policy_version 72660 (0.0008) -[2023-10-15 17:40:51,869][52866] Updated weights for policy 1, policy_version 72670 (0.0008) -[2023-10-15 17:40:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 148570112. Throughput: 0: 1786.3, 1: 1797.6. Samples: 37150522. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:53,442][51532] Avg episode reward: [(0, '61.310'), (1, '61.860')] -[2023-10-15 17:40:53,738][52833] Updated weights for policy 0, policy_version 72420 (0.0008) -[2023-10-15 17:40:54,134][52833] Updated weights for policy 0, policy_version 72430 (0.0008) -[2023-10-15 17:40:54,488][52833] Updated weights for policy 0, policy_version 72440 (0.0008) -[2023-10-15 17:40:55,476][52866] Updated weights for policy 1, policy_version 72680 (0.0009) -[2023-10-15 17:40:55,852][52866] Updated weights for policy 1, policy_version 72690 (0.0009) -[2023-10-15 17:40:56,230][52866] Updated weights for policy 1, policy_version 72700 (0.0009) -[2023-10-15 17:40:58,245][52833] Updated weights for policy 0, policy_version 72450 (0.0010) -[2023-10-15 17:40:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148635648. Throughput: 0: 1790.0, 1: 1794.4. Samples: 37172852. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:40:58,442][51532] Avg episode reward: [(0, '58.890'), (1, '61.150')] -[2023-10-15 17:40:58,611][52833] Updated weights for policy 0, policy_version 72460 (0.0008) -[2023-10-15 17:40:58,979][52833] Updated weights for policy 0, policy_version 72470 (0.0007) -[2023-10-15 17:40:59,354][52833] Updated weights for policy 0, policy_version 72480 (0.0009) -[2023-10-15 17:40:59,753][52866] Updated weights for policy 1, policy_version 72710 (0.0007) -[2023-10-15 17:41:00,125][52866] Updated weights for policy 1, policy_version 72720 (0.0007) -[2023-10-15 17:41:00,490][52866] Updated weights for policy 1, policy_version 72730 (0.0007) -[2023-10-15 17:41:03,129][52833] Updated weights for policy 0, policy_version 72490 (0.0010) -[2023-10-15 17:41:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 148701184. Throughput: 0: 1786.0, 1: 1804.0. Samples: 37182930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:03,442][51532] Avg episode reward: [(0, '56.550'), (1, '59.890')] -[2023-10-15 17:41:03,504][52833] Updated weights for policy 0, policy_version 72500 (0.0007) -[2023-10-15 17:41:03,879][52833] Updated weights for policy 0, policy_version 72510 (0.0009) -[2023-10-15 17:41:04,259][52866] Updated weights for policy 1, policy_version 72740 (0.0008) -[2023-10-15 17:41:04,628][52866] Updated weights for policy 1, policy_version 72750 (0.0010) -[2023-10-15 17:41:04,999][52866] Updated weights for policy 1, policy_version 72760 (0.0009) -[2023-10-15 17:41:07,558][52833] Updated weights for policy 0, policy_version 72520 (0.0008) -[2023-10-15 17:41:07,932][52833] Updated weights for policy 0, policy_version 72530 (0.0009) -[2023-10-15 17:41:08,306][52833] Updated weights for policy 0, policy_version 72540 (0.0009) -[2023-10-15 17:41:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 148766720. Throughput: 0: 1787.6, 1: 1806.9. Samples: 37205490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:08,442][51532] Avg episode reward: [(0, '57.260'), (1, '62.750')] -[2023-10-15 17:41:08,710][52866] Updated weights for policy 1, policy_version 72770 (0.0009) -[2023-10-15 17:41:09,089][52866] Updated weights for policy 1, policy_version 72780 (0.0010) -[2023-10-15 17:41:09,458][52866] Updated weights for policy 1, policy_version 72790 (0.0010) -[2023-10-15 17:41:09,819][52866] Updated weights for policy 1, policy_version 72800 (0.0010) -[2023-10-15 17:41:12,110][52833] Updated weights for policy 0, policy_version 72550 (0.0007) -[2023-10-15 17:41:12,478][52833] Updated weights for policy 0, policy_version 72560 (0.0007) -[2023-10-15 17:41:12,842][52833] Updated weights for policy 0, policy_version 72570 (0.0008) -[2023-10-15 17:41:13,357][52866] Updated weights for policy 1, policy_version 72810 (0.0008) -[2023-10-15 17:41:13,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 148865024. Throughput: 0: 1792.9, 1: 1816.9. Samples: 37227148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:13,442][51532] Avg episode reward: [(0, '57.960'), (1, '62.500')] -[2023-10-15 17:41:13,730][52866] Updated weights for policy 1, policy_version 72820 (0.0009) -[2023-10-15 17:41:14,101][52866] Updated weights for policy 1, policy_version 72830 (0.0007) -[2023-10-15 17:41:16,518][52833] Updated weights for policy 0, policy_version 72580 (0.0007) -[2023-10-15 17:41:16,887][52833] Updated weights for policy 0, policy_version 72590 (0.0007) -[2023-10-15 17:41:17,253][52833] Updated weights for policy 0, policy_version 72600 (0.0007) -[2023-10-15 17:41:17,843][52866] Updated weights for policy 1, policy_version 72840 (0.0007) -[2023-10-15 17:41:18,204][52866] Updated weights for policy 1, policy_version 72850 (0.0008) -[2023-10-15 17:41:18,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 148930560. Throughput: 0: 1791.1, 1: 1812.0. Samples: 37238306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:18,441][51532] Avg episode reward: [(0, '61.670'), (1, '62.190')] -[2023-10-15 17:41:18,570][52866] Updated weights for policy 1, policy_version 72860 (0.0010) -[2023-10-15 17:41:21,070][52833] Updated weights for policy 0, policy_version 72610 (0.0008) -[2023-10-15 17:41:21,446][52833] Updated weights for policy 0, policy_version 72620 (0.0011) -[2023-10-15 17:41:21,820][52833] Updated weights for policy 0, policy_version 72630 (0.0011) -[2023-10-15 17:41:22,185][52833] Updated weights for policy 0, policy_version 72640 (0.0009) -[2023-10-15 17:41:22,421][52866] Updated weights for policy 1, policy_version 72870 (0.0011) -[2023-10-15 17:41:22,795][52866] Updated weights for policy 1, policy_version 72880 (0.0011) -[2023-10-15 17:41:23,162][52866] Updated weights for policy 1, policy_version 72890 (0.0010) -[2023-10-15 17:41:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149028864. Throughput: 0: 1799.0, 1: 1809.1. Samples: 37259676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:23,442][51532] Avg episode reward: [(0, '63.310'), (1, '61.340')] -[2023-10-15 17:41:25,950][52833] Updated weights for policy 0, policy_version 72650 (0.0010) -[2023-10-15 17:41:26,315][52833] Updated weights for policy 0, policy_version 72660 (0.0010) -[2023-10-15 17:41:26,678][52833] Updated weights for policy 0, policy_version 72670 (0.0008) -[2023-10-15 17:41:26,883][52866] Updated weights for policy 1, policy_version 72900 (0.0010) -[2023-10-15 17:41:27,257][52866] Updated weights for policy 1, policy_version 72910 (0.0009) -[2023-10-15 17:41:27,617][52866] Updated weights for policy 1, policy_version 72920 (0.0011) -[2023-10-15 17:41:28,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149094400. Throughput: 0: 1794.7, 1: 1811.0. Samples: 37280612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:28,442][51532] Avg episode reward: [(0, '62.970'), (1, '59.990')] -[2023-10-15 17:41:30,354][52833] Updated weights for policy 0, policy_version 72680 (0.0007) -[2023-10-15 17:41:30,725][52833] Updated weights for policy 0, policy_version 72690 (0.0007) -[2023-10-15 17:41:31,095][52833] Updated weights for policy 0, policy_version 72700 (0.0007) -[2023-10-15 17:41:31,361][52866] Updated weights for policy 1, policy_version 72930 (0.0011) -[2023-10-15 17:41:31,732][52866] Updated weights for policy 1, policy_version 72940 (0.0009) -[2023-10-15 17:41:32,097][52866] Updated weights for policy 1, policy_version 72950 (0.0007) -[2023-10-15 17:41:32,464][52866] Updated weights for policy 1, policy_version 72960 (0.0008) -[2023-10-15 17:41:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149159936. Throughput: 0: 1807.8, 1: 1803.8. Samples: 37292234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:33,441][51532] Avg episode reward: [(0, '62.490'), (1, '59.170')] -[2023-10-15 17:41:34,710][52833] Updated weights for policy 0, policy_version 72710 (0.0009) -[2023-10-15 17:41:35,078][52833] Updated weights for policy 0, policy_version 72720 (0.0010) -[2023-10-15 17:41:35,454][52833] Updated weights for policy 0, policy_version 72730 (0.0008) -[2023-10-15 17:41:36,271][52866] Updated weights for policy 1, policy_version 72970 (0.0008) -[2023-10-15 17:41:36,641][52866] Updated weights for policy 1, policy_version 72980 (0.0007) -[2023-10-15 17:41:37,005][52866] Updated weights for policy 1, policy_version 72990 (0.0009) -[2023-10-15 17:41:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149225472. Throughput: 0: 1800.3, 1: 1812.0. Samples: 37313072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:38,442][51532] Avg episode reward: [(0, '60.970'), (1, '60.660')] -[2023-10-15 17:41:39,293][52833] Updated weights for policy 0, policy_version 72740 (0.0009) -[2023-10-15 17:41:39,679][52833] Updated weights for policy 0, policy_version 72750 (0.0008) -[2023-10-15 17:41:40,054][52833] Updated weights for policy 0, policy_version 72760 (0.0007) -[2023-10-15 17:41:40,682][52866] Updated weights for policy 1, policy_version 73000 (0.0007) -[2023-10-15 17:41:41,047][52866] Updated weights for policy 1, policy_version 73010 (0.0007) -[2023-10-15 17:41:41,421][52866] Updated weights for policy 1, policy_version 73020 (0.0009) -[2023-10-15 17:41:43,441][51532] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 149291008. Throughput: 0: 1803.8, 1: 1812.2. Samples: 37335572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:43,442][51532] Avg episode reward: [(0, '62.280'), (1, '64.370')] -[2023-10-15 17:41:43,453][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000072768_74514432.pth... -[2023-10-15 17:41:43,453][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000073024_74776576.pth... -[2023-10-15 17:41:43,483][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000071104_72810496.pth -[2023-10-15 17:41:43,494][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000071328_73039872.pth -[2023-10-15 17:41:43,683][52833] Updated weights for policy 0, policy_version 72770 (0.0009) -[2023-10-15 17:41:44,059][52833] Updated weights for policy 0, policy_version 72780 (0.0008) -[2023-10-15 17:41:44,436][52833] Updated weights for policy 0, policy_version 72790 (0.0009) -[2023-10-15 17:41:44,813][52833] Updated weights for policy 0, policy_version 72800 (0.0008) -[2023-10-15 17:41:45,196][52866] Updated weights for policy 1, policy_version 73030 (0.0009) -[2023-10-15 17:41:45,580][52866] Updated weights for policy 1, policy_version 73040 (0.0007) -[2023-10-15 17:41:45,941][52866] Updated weights for policy 1, policy_version 73050 (0.0007) -[2023-10-15 17:41:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149356544. Throughput: 0: 1808.8, 1: 1810.4. Samples: 37345794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:48,441][51532] Avg episode reward: [(0, '64.480'), (1, '68.940')] -[2023-10-15 17:41:48,534][52833] Updated weights for policy 0, policy_version 72810 (0.0008) -[2023-10-15 17:41:48,897][52833] Updated weights for policy 0, policy_version 72820 (0.0010) -[2023-10-15 17:41:49,264][52833] Updated weights for policy 0, policy_version 72830 (0.0008) -[2023-10-15 17:41:49,683][52866] Updated weights for policy 1, policy_version 73060 (0.0009) -[2023-10-15 17:41:50,051][52866] Updated weights for policy 1, policy_version 73070 (0.0008) -[2023-10-15 17:41:50,413][52866] Updated weights for policy 1, policy_version 73080 (0.0008) -[2023-10-15 17:41:52,917][52833] Updated weights for policy 0, policy_version 72840 (0.0010) -[2023-10-15 17:41:53,278][52833] Updated weights for policy 0, policy_version 72850 (0.0009) -[2023-10-15 17:41:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149422080. Throughput: 0: 1805.6, 1: 1798.8. Samples: 37367690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:53,442][51532] Avg episode reward: [(0, '63.880'), (1, '69.200')] -[2023-10-15 17:41:53,655][52833] Updated weights for policy 0, policy_version 72860 (0.0009) -[2023-10-15 17:41:54,066][52866] Updated weights for policy 1, policy_version 73090 (0.0007) -[2023-10-15 17:41:54,427][52866] Updated weights for policy 1, policy_version 73100 (0.0007) -[2023-10-15 17:41:54,792][52866] Updated weights for policy 1, policy_version 73110 (0.0010) -[2023-10-15 17:41:55,154][52866] Updated weights for policy 1, policy_version 73120 (0.0009) -[2023-10-15 17:41:57,477][52833] Updated weights for policy 0, policy_version 72870 (0.0008) -[2023-10-15 17:41:57,841][52833] Updated weights for policy 0, policy_version 72880 (0.0007) -[2023-10-15 17:41:58,210][52833] Updated weights for policy 0, policy_version 72890 (0.0008) -[2023-10-15 17:41:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149520384. Throughput: 0: 1813.9, 1: 1793.8. Samples: 37389492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:41:58,441][51532] Avg episode reward: [(0, '63.650'), (1, '71.950')] -[2023-10-15 17:41:58,880][52866] Updated weights for policy 1, policy_version 73130 (0.0007) -[2023-10-15 17:41:59,253][52866] Updated weights for policy 1, policy_version 73140 (0.0009) -[2023-10-15 17:41:59,622][52866] Updated weights for policy 1, policy_version 73150 (0.0007) -[2023-10-15 17:42:02,162][52833] Updated weights for policy 0, policy_version 72900 (0.0009) -[2023-10-15 17:42:02,541][52833] Updated weights for policy 0, policy_version 72910 (0.0007) -[2023-10-15 17:42:02,908][52833] Updated weights for policy 0, policy_version 72920 (0.0008) -[2023-10-15 17:42:03,251][52866] Updated weights for policy 1, policy_version 73160 (0.0009) -[2023-10-15 17:42:03,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149585920. Throughput: 0: 1799.4, 1: 1796.8. Samples: 37400138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:42:03,442][51532] Avg episode reward: [(0, '60.600'), (1, '74.520')] -[2023-10-15 17:42:03,608][52866] Updated weights for policy 1, policy_version 73170 (0.0009) -[2023-10-15 17:42:03,981][52866] Updated weights for policy 1, policy_version 73180 (0.0009) -[2023-10-15 17:42:06,607][52833] Updated weights for policy 0, policy_version 72930 (0.0008) -[2023-10-15 17:42:06,972][52833] Updated weights for policy 0, policy_version 72940 (0.0008) -[2023-10-15 17:42:07,334][52833] Updated weights for policy 0, policy_version 72950 (0.0007) -[2023-10-15 17:42:07,695][52833] Updated weights for policy 0, policy_version 72960 (0.0009) -[2023-10-15 17:42:07,770][52866] Updated weights for policy 1, policy_version 73190 (0.0008) -[2023-10-15 17:42:08,131][52866] Updated weights for policy 1, policy_version 73200 (0.0009) -[2023-10-15 17:42:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149651456. Throughput: 0: 1812.4, 1: 1801.8. Samples: 37422314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:42:08,442][51532] Avg episode reward: [(0, '61.090'), (1, '74.130')] -[2023-10-15 17:42:08,501][52866] Updated weights for policy 1, policy_version 73210 (0.0008) -[2023-10-15 17:42:11,243][52833] Updated weights for policy 0, policy_version 72970 (0.0009) -[2023-10-15 17:42:11,603][52833] Updated weights for policy 0, policy_version 72980 (0.0008) -[2023-10-15 17:42:11,975][52833] Updated weights for policy 0, policy_version 72990 (0.0010) -[2023-10-15 17:42:12,265][52866] Updated weights for policy 1, policy_version 73220 (0.0008) -[2023-10-15 17:42:12,630][52866] Updated weights for policy 1, policy_version 73230 (0.0007) -[2023-10-15 17:42:12,989][52866] Updated weights for policy 1, policy_version 73240 (0.0007) -[2023-10-15 17:42:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149749760. Throughput: 0: 1801.0, 1: 1809.7. Samples: 37443094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:42:13,441][51532] Avg episode reward: [(0, '60.800'), (1, '72.230')] -[2023-10-15 17:42:15,782][52833] Updated weights for policy 0, policy_version 73000 (0.0009) -[2023-10-15 17:42:16,148][52833] Updated weights for policy 0, policy_version 73010 (0.0008) -[2023-10-15 17:42:16,529][52833] Updated weights for policy 0, policy_version 73020 (0.0008) -[2023-10-15 17:42:16,757][52866] Updated weights for policy 1, policy_version 73250 (0.0007) -[2023-10-15 17:42:17,129][52866] Updated weights for policy 1, policy_version 73260 (0.0011) -[2023-10-15 17:42:17,491][52866] Updated weights for policy 1, policy_version 73270 (0.0009) -[2023-10-15 17:42:17,861][52866] Updated weights for policy 1, policy_version 73280 (0.0012) -[2023-10-15 17:42:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 149815296. Throughput: 0: 1816.6, 1: 1803.5. Samples: 37455138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:42:18,442][51532] Avg episode reward: [(0, '63.120'), (1, '71.700')] -[2023-10-15 17:42:20,125][52833] Updated weights for policy 0, policy_version 73030 (0.0007) -[2023-10-15 17:42:20,492][52833] Updated weights for policy 0, policy_version 73040 (0.0007) -[2023-10-15 17:42:20,862][52833] Updated weights for policy 0, policy_version 73050 (0.0007) -[2023-10-15 17:42:21,592][52866] Updated weights for policy 1, policy_version 73290 (0.0010) -[2023-10-15 17:42:21,954][52866] Updated weights for policy 1, policy_version 73300 (0.0008) -[2023-10-15 17:42:22,317][52866] Updated weights for policy 1, policy_version 73310 (0.0007) -[2023-10-15 17:42:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 149880832. Throughput: 0: 1804.3, 1: 1810.3. Samples: 37475726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:42:23,442][51532] Avg episode reward: [(0, '62.610'), (1, '70.430')] -[2023-10-15 17:42:24,678][52833] Updated weights for policy 0, policy_version 73060 (0.0008) -[2023-10-15 17:42:25,062][52833] Updated weights for policy 0, policy_version 73070 (0.0007) -[2023-10-15 17:42:25,437][52833] Updated weights for policy 0, policy_version 73080 (0.0008) -[2023-10-15 17:42:26,148][52866] Updated weights for policy 1, policy_version 73320 (0.0009) -[2023-10-15 17:42:26,519][52866] Updated weights for policy 1, policy_version 73330 (0.0008) -[2023-10-15 17:42:26,888][52866] Updated weights for policy 1, policy_version 73340 (0.0008) -[2023-10-15 17:42:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149946368. Throughput: 0: 1806.8, 1: 1793.4. Samples: 37497582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:42:28,442][51532] Avg episode reward: [(0, '61.420'), (1, '71.340')] -[2023-10-15 17:42:29,063][52833] Updated weights for policy 0, policy_version 73090 (0.0009) -[2023-10-15 17:42:29,425][52833] Updated weights for policy 0, policy_version 73100 (0.0009) -[2023-10-15 17:42:29,799][52833] Updated weights for policy 0, policy_version 73110 (0.0008) -[2023-10-15 17:42:30,164][52833] Updated weights for policy 0, policy_version 73120 (0.0009) -[2023-10-15 17:42:30,586][52866] Updated weights for policy 1, policy_version 73350 (0.0010) -[2023-10-15 17:42:30,954][52866] Updated weights for policy 1, policy_version 73360 (0.0009) -[2023-10-15 17:42:31,320][52866] Updated weights for policy 1, policy_version 73370 (0.0011) -[2023-10-15 17:42:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150011904. Throughput: 0: 1802.8, 1: 1804.6. Samples: 37508124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:42:33,441][51532] Avg episode reward: [(0, '59.860'), (1, '71.160')] -[2023-10-15 17:42:33,950][52833] Updated weights for policy 0, policy_version 73130 (0.0011) -[2023-10-15 17:42:34,319][52833] Updated weights for policy 0, policy_version 73140 (0.0008) -[2023-10-15 17:42:34,679][52833] Updated weights for policy 0, policy_version 73150 (0.0009) -[2023-10-15 17:42:35,051][52866] Updated weights for policy 1, policy_version 73380 (0.0008) -[2023-10-15 17:42:35,415][52866] Updated weights for policy 1, policy_version 73390 (0.0007) -[2023-10-15 17:42:35,784][52866] Updated weights for policy 1, policy_version 73400 (0.0009) -[2023-10-15 17:42:38,288][52833] Updated weights for policy 0, policy_version 73160 (0.0008) -[2023-10-15 17:42:38,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150077440. Throughput: 0: 1812.5, 1: 1798.3. Samples: 37530178. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:42:38,441][51532] Avg episode reward: [(0, '56.730'), (1, '67.000')] -[2023-10-15 17:42:38,654][52833] Updated weights for policy 0, policy_version 73170 (0.0008) -[2023-10-15 17:42:39,019][52833] Updated weights for policy 0, policy_version 73180 (0.0007) -[2023-10-15 17:42:39,443][52866] Updated weights for policy 1, policy_version 73410 (0.0007) -[2023-10-15 17:42:39,813][52866] Updated weights for policy 1, policy_version 73420 (0.0010) -[2023-10-15 17:42:40,175][52866] Updated weights for policy 1, policy_version 73430 (0.0011) -[2023-10-15 17:42:40,546][52866] Updated weights for policy 1, policy_version 73440 (0.0010) -[2023-10-15 17:42:42,783][52833] Updated weights for policy 0, policy_version 73190 (0.0009) -[2023-10-15 17:42:43,149][52833] Updated weights for policy 0, policy_version 73200 (0.0008) -[2023-10-15 17:42:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150142976. Throughput: 0: 1819.7, 1: 1802.9. Samples: 37552510. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:42:43,442][51532] Avg episode reward: [(0, '57.770'), (1, '59.390')] -[2023-10-15 17:42:43,517][52833] Updated weights for policy 0, policy_version 73210 (0.0007) -[2023-10-15 17:42:44,136][52866] Updated weights for policy 1, policy_version 73450 (0.0008) -[2023-10-15 17:42:44,503][52866] Updated weights for policy 1, policy_version 73460 (0.0009) -[2023-10-15 17:42:44,867][52866] Updated weights for policy 1, policy_version 73470 (0.0008) -[2023-10-15 17:42:47,282][52833] Updated weights for policy 0, policy_version 73220 (0.0009) -[2023-10-15 17:42:47,656][52833] Updated weights for policy 0, policy_version 73230 (0.0008) -[2023-10-15 17:42:48,032][52833] Updated weights for policy 0, policy_version 73240 (0.0008) -[2023-10-15 17:42:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 150241280. Throughput: 0: 1812.8, 1: 1798.0. Samples: 37562628. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:42:48,442][51532] Avg episode reward: [(0, '59.470'), (1, '58.040')] -[2023-10-15 17:42:48,632][52866] Updated weights for policy 1, policy_version 73480 (0.0008) -[2023-10-15 17:42:48,999][52866] Updated weights for policy 1, policy_version 73490 (0.0007) -[2023-10-15 17:42:49,366][52866] Updated weights for policy 1, policy_version 73500 (0.0007) -[2023-10-15 17:42:51,869][52833] Updated weights for policy 0, policy_version 73250 (0.0009) -[2023-10-15 17:42:52,250][52833] Updated weights for policy 0, policy_version 73260 (0.0008) -[2023-10-15 17:42:52,615][52833] Updated weights for policy 0, policy_version 73270 (0.0010) -[2023-10-15 17:42:52,993][52833] Updated weights for policy 0, policy_version 73280 (0.0009) -[2023-10-15 17:42:53,053][52866] Updated weights for policy 1, policy_version 73510 (0.0008) -[2023-10-15 17:42:53,413][52866] Updated weights for policy 1, policy_version 73520 (0.0010) -[2023-10-15 17:42:53,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 150306816. Throughput: 0: 1817.1, 1: 1800.0. Samples: 37585082. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:42:53,442][51532] Avg episode reward: [(0, '64.160'), (1, '57.350')] -[2023-10-15 17:42:53,788][52866] Updated weights for policy 1, policy_version 73530 (0.0007) -[2023-10-15 17:42:56,554][52833] Updated weights for policy 0, policy_version 73290 (0.0010) -[2023-10-15 17:42:56,922][52833] Updated weights for policy 0, policy_version 73300 (0.0007) -[2023-10-15 17:42:57,285][52833] Updated weights for policy 0, policy_version 73310 (0.0009) -[2023-10-15 17:42:57,531][52866] Updated weights for policy 1, policy_version 73540 (0.0008) -[2023-10-15 17:42:57,899][52866] Updated weights for policy 1, policy_version 73550 (0.0010) -[2023-10-15 17:42:58,270][52866] Updated weights for policy 1, policy_version 73560 (0.0008) -[2023-10-15 17:42:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 150372352. Throughput: 0: 1805.9, 1: 1806.6. Samples: 37605658. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:42:58,442][51532] Avg episode reward: [(0, '64.340'), (1, '59.460')] -[2023-10-15 17:43:01,026][52833] Updated weights for policy 0, policy_version 73320 (0.0010) -[2023-10-15 17:43:01,403][52833] Updated weights for policy 0, policy_version 73330 (0.0010) -[2023-10-15 17:43:01,770][52833] Updated weights for policy 0, policy_version 73340 (0.0010) -[2023-10-15 17:43:01,950][52866] Updated weights for policy 1, policy_version 73570 (0.0009) -[2023-10-15 17:43:02,324][52866] Updated weights for policy 1, policy_version 73580 (0.0010) -[2023-10-15 17:43:02,691][52866] Updated weights for policy 1, policy_version 73590 (0.0010) -[2023-10-15 17:43:03,056][52866] Updated weights for policy 1, policy_version 73600 (0.0008) -[2023-10-15 17:43:03,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 150470656. Throughput: 0: 1808.5, 1: 1804.9. Samples: 37617744. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:43:03,442][51532] Avg episode reward: [(0, '62.870'), (1, '60.980')] -[2023-10-15 17:43:05,389][52833] Updated weights for policy 0, policy_version 73350 (0.0010) -[2023-10-15 17:43:05,752][52833] Updated weights for policy 0, policy_version 73360 (0.0010) -[2023-10-15 17:43:06,116][52833] Updated weights for policy 0, policy_version 73370 (0.0009) -[2023-10-15 17:43:06,752][52866] Updated weights for policy 1, policy_version 73610 (0.0009) -[2023-10-15 17:43:07,118][52866] Updated weights for policy 1, policy_version 73620 (0.0008) -[2023-10-15 17:43:07,487][52866] Updated weights for policy 1, policy_version 73630 (0.0009) -[2023-10-15 17:43:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 150536192. Throughput: 0: 1798.8, 1: 1812.4. Samples: 37638226. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:43:08,442][51532] Avg episode reward: [(0, '64.270'), (1, '56.540')] -[2023-10-15 17:43:10,015][52833] Updated weights for policy 0, policy_version 73380 (0.0008) -[2023-10-15 17:43:10,402][52833] Updated weights for policy 0, policy_version 73390 (0.0007) -[2023-10-15 17:43:10,765][52833] Updated weights for policy 0, policy_version 73400 (0.0008) -[2023-10-15 17:43:11,311][52866] Updated weights for policy 1, policy_version 73640 (0.0011) -[2023-10-15 17:43:11,672][52866] Updated weights for policy 1, policy_version 73650 (0.0010) -[2023-10-15 17:43:12,028][52866] Updated weights for policy 1, policy_version 73660 (0.0007) -[2023-10-15 17:43:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 150601728. Throughput: 0: 1794.7, 1: 1810.5. Samples: 37659812. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:43:13,442][51532] Avg episode reward: [(0, '63.130'), (1, '55.570')] -[2023-10-15 17:43:14,453][52833] Updated weights for policy 0, policy_version 73410 (0.0008) -[2023-10-15 17:43:14,826][52833] Updated weights for policy 0, policy_version 73420 (0.0008) -[2023-10-15 17:43:15,199][52833] Updated weights for policy 0, policy_version 73430 (0.0009) -[2023-10-15 17:43:15,566][52833] Updated weights for policy 0, policy_version 73440 (0.0009) -[2023-10-15 17:43:15,839][52866] Updated weights for policy 1, policy_version 73670 (0.0007) -[2023-10-15 17:43:16,217][52866] Updated weights for policy 1, policy_version 73680 (0.0007) -[2023-10-15 17:43:16,591][52866] Updated weights for policy 1, policy_version 73690 (0.0010) -[2023-10-15 17:43:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150667264. Throughput: 0: 1793.5, 1: 1820.7. Samples: 37670766. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:43:18,442][51532] Avg episode reward: [(0, '63.570'), (1, '58.740')] -[2023-10-15 17:43:19,350][52833] Updated weights for policy 0, policy_version 73450 (0.0008) -[2023-10-15 17:43:19,717][52833] Updated weights for policy 0, policy_version 73460 (0.0007) -[2023-10-15 17:43:20,086][52833] Updated weights for policy 0, policy_version 73470 (0.0007) -[2023-10-15 17:43:20,278][52866] Updated weights for policy 1, policy_version 73700 (0.0011) -[2023-10-15 17:43:20,645][52866] Updated weights for policy 1, policy_version 73710 (0.0008) -[2023-10-15 17:43:21,005][52866] Updated weights for policy 1, policy_version 73720 (0.0009) -[2023-10-15 17:43:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150732800. Throughput: 0: 1789.5, 1: 1815.6. Samples: 37692408. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-15 17:43:23,442][51532] Avg episode reward: [(0, '65.640'), (1, '60.340')] -[2023-10-15 17:43:23,927][52833] Updated weights for policy 0, policy_version 73480 (0.0007) -[2023-10-15 17:43:24,290][52833] Updated weights for policy 0, policy_version 73490 (0.0007) -[2023-10-15 17:43:24,660][52833] Updated weights for policy 0, policy_version 73500 (0.0007) -[2023-10-15 17:43:24,715][52866] Updated weights for policy 1, policy_version 73730 (0.0009) -[2023-10-15 17:43:25,084][52866] Updated weights for policy 1, policy_version 73740 (0.0009) -[2023-10-15 17:43:25,453][52866] Updated weights for policy 1, policy_version 73750 (0.0008) -[2023-10-15 17:43:25,821][52866] Updated weights for policy 1, policy_version 73760 (0.0008) -[2023-10-15 17:43:28,408][52833] Updated weights for policy 0, policy_version 73510 (0.0007) -[2023-10-15 17:43:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150798336. Throughput: 0: 1794.9, 1: 1813.7. Samples: 37714894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:43:28,442][51532] Avg episode reward: [(0, '64.880'), (1, '60.400')] -[2023-10-15 17:43:28,783][52833] Updated weights for policy 0, policy_version 73520 (0.0008) -[2023-10-15 17:43:29,148][52833] Updated weights for policy 0, policy_version 73530 (0.0008) -[2023-10-15 17:43:29,427][52866] Updated weights for policy 1, policy_version 73770 (0.0007) -[2023-10-15 17:43:29,790][52866] Updated weights for policy 1, policy_version 73780 (0.0011) -[2023-10-15 17:43:30,159][52866] Updated weights for policy 1, policy_version 73790 (0.0007) -[2023-10-15 17:43:33,045][52833] Updated weights for policy 0, policy_version 73540 (0.0008) -[2023-10-15 17:43:33,425][52833] Updated weights for policy 0, policy_version 73550 (0.0010) -[2023-10-15 17:43:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150863872. Throughput: 0: 1786.4, 1: 1816.3. Samples: 37724752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:43:33,441][51532] Avg episode reward: [(0, '66.890'), (1, '61.290')] -[2023-10-15 17:43:33,786][52833] Updated weights for policy 0, policy_version 73560 (0.0008) -[2023-10-15 17:43:33,855][52866] Updated weights for policy 1, policy_version 73800 (0.0007) -[2023-10-15 17:43:34,219][52866] Updated weights for policy 1, policy_version 73810 (0.0007) -[2023-10-15 17:43:34,583][52866] Updated weights for policy 1, policy_version 73820 (0.0010) -[2023-10-15 17:43:37,574][52833] Updated weights for policy 0, policy_version 73570 (0.0007) -[2023-10-15 17:43:37,954][52833] Updated weights for policy 0, policy_version 73580 (0.0010) -[2023-10-15 17:43:38,319][52833] Updated weights for policy 0, policy_version 73590 (0.0009) -[2023-10-15 17:43:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 150929408. Throughput: 0: 1789.0, 1: 1809.6. Samples: 37747022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:43:38,442][51532] Avg episode reward: [(0, '68.830'), (1, '62.540')] -[2023-10-15 17:43:38,466][52866] Updated weights for policy 1, policy_version 73830 (0.0009) -[2023-10-15 17:43:38,692][52833] Updated weights for policy 0, policy_version 73600 (0.0007) -[2023-10-15 17:43:38,832][52866] Updated weights for policy 1, policy_version 73840 (0.0008) -[2023-10-15 17:43:39,203][52866] Updated weights for policy 1, policy_version 73850 (0.0007) -[2023-10-15 17:43:42,524][52833] Updated weights for policy 0, policy_version 73610 (0.0008) -[2023-10-15 17:43:42,890][52833] Updated weights for policy 0, policy_version 73620 (0.0007) -[2023-10-15 17:43:43,031][52866] Updated weights for policy 1, policy_version 73860 (0.0008) -[2023-10-15 17:43:43,257][52833] Updated weights for policy 0, policy_version 73630 (0.0007) -[2023-10-15 17:43:43,389][52866] Updated weights for policy 1, policy_version 73870 (0.0007) -[2023-10-15 17:43:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 151027712. Throughput: 0: 1793.1, 1: 1820.3. Samples: 37768260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:43:43,442][51532] Avg episode reward: [(0, '71.250'), (1, '63.600')] -[2023-10-15 17:43:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000073632_75399168.pth... -[2023-10-15 17:43:43,486][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000071936_73662464.pth -[2023-10-15 17:43:43,490][52410] Saving new best policy, reward=71.250! -[2023-10-15 17:43:43,758][52866] Updated weights for policy 1, policy_version 73880 (0.0007) -[2023-10-15 17:43:44,046][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000073888_75661312.pth... -[2023-10-15 17:43:44,084][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000072192_73924608.pth -[2023-10-15 17:43:46,877][52833] Updated weights for policy 0, policy_version 73640 (0.0009) -[2023-10-15 17:43:47,256][52833] Updated weights for policy 0, policy_version 73650 (0.0010) -[2023-10-15 17:43:47,583][52866] Updated weights for policy 1, policy_version 73890 (0.0008) -[2023-10-15 17:43:47,624][52833] Updated weights for policy 0, policy_version 73660 (0.0010) -[2023-10-15 17:43:47,948][52866] Updated weights for policy 1, policy_version 73900 (0.0008) -[2023-10-15 17:43:48,323][52866] Updated weights for policy 1, policy_version 73910 (0.0009) -[2023-10-15 17:43:48,441][51532] Fps is (10 sec: 16384.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151093248. Throughput: 0: 1788.1, 1: 1798.2. Samples: 37779126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:43:48,441][51532] Avg episode reward: [(0, '72.210'), (1, '61.970')] -[2023-10-15 17:43:48,442][52410] Saving new best policy, reward=72.210! -[2023-10-15 17:43:48,689][52866] Updated weights for policy 1, policy_version 73920 (0.0009) -[2023-10-15 17:43:51,332][52833] Updated weights for policy 0, policy_version 73670 (0.0009) -[2023-10-15 17:43:51,702][52833] Updated weights for policy 0, policy_version 73680 (0.0008) -[2023-10-15 17:43:52,083][52833] Updated weights for policy 0, policy_version 73690 (0.0009) -[2023-10-15 17:43:52,579][52866] Updated weights for policy 1, policy_version 73930 (0.0008) -[2023-10-15 17:43:52,934][52866] Updated weights for policy 1, policy_version 73940 (0.0008) -[2023-10-15 17:43:53,310][52866] Updated weights for policy 1, policy_version 73950 (0.0008) -[2023-10-15 17:43:53,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 151191552. Throughput: 0: 1800.0, 1: 1809.4. Samples: 37800648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:43:53,441][51532] Avg episode reward: [(0, '69.870'), (1, '60.830')] -[2023-10-15 17:43:55,814][52833] Updated weights for policy 0, policy_version 73700 (0.0007) -[2023-10-15 17:43:56,194][52833] Updated weights for policy 0, policy_version 73710 (0.0007) -[2023-10-15 17:43:56,560][52833] Updated weights for policy 0, policy_version 73720 (0.0008) -[2023-10-15 17:43:56,953][52866] Updated weights for policy 1, policy_version 73960 (0.0009) -[2023-10-15 17:43:57,317][52866] Updated weights for policy 1, policy_version 73970 (0.0009) -[2023-10-15 17:43:57,695][52866] Updated weights for policy 1, policy_version 73980 (0.0010) -[2023-10-15 17:43:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 151257088. Throughput: 0: 1785.1, 1: 1799.9. Samples: 37821138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:43:58,442][51532] Avg episode reward: [(0, '71.090'), (1, '58.640')] -[2023-10-15 17:44:00,456][52833] Updated weights for policy 0, policy_version 73730 (0.0008) -[2023-10-15 17:44:00,819][52833] Updated weights for policy 0, policy_version 73740 (0.0008) -[2023-10-15 17:44:01,191][52833] Updated weights for policy 0, policy_version 73750 (0.0007) -[2023-10-15 17:44:01,498][52866] Updated weights for policy 1, policy_version 73990 (0.0010) -[2023-10-15 17:44:01,561][52833] Updated weights for policy 0, policy_version 73760 (0.0007) -[2023-10-15 17:44:01,881][52866] Updated weights for policy 1, policy_version 74000 (0.0009) -[2023-10-15 17:44:02,252][52866] Updated weights for policy 1, policy_version 74010 (0.0007) -[2023-10-15 17:44:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151322624. Throughput: 0: 1802.6, 1: 1805.9. Samples: 37833148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:03,442][51532] Avg episode reward: [(0, '69.620'), (1, '60.510')] -[2023-10-15 17:44:05,202][52833] Updated weights for policy 0, policy_version 73770 (0.0007) -[2023-10-15 17:44:05,571][52833] Updated weights for policy 0, policy_version 73780 (0.0009) -[2023-10-15 17:44:05,945][52833] Updated weights for policy 0, policy_version 73790 (0.0008) -[2023-10-15 17:44:05,948][52866] Updated weights for policy 1, policy_version 74020 (0.0008) -[2023-10-15 17:44:06,316][52866] Updated weights for policy 1, policy_version 74030 (0.0008) -[2023-10-15 17:44:06,680][52866] Updated weights for policy 1, policy_version 74040 (0.0009) -[2023-10-15 17:44:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151388160. Throughput: 0: 1781.6, 1: 1793.6. Samples: 37853296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:08,442][51532] Avg episode reward: [(0, '68.290'), (1, '65.460')] -[2023-10-15 17:44:09,732][52833] Updated weights for policy 0, policy_version 73800 (0.0010) -[2023-10-15 17:44:10,110][52833] Updated weights for policy 0, policy_version 73810 (0.0010) -[2023-10-15 17:44:10,318][52866] Updated weights for policy 1, policy_version 74050 (0.0007) -[2023-10-15 17:44:10,477][52833] Updated weights for policy 0, policy_version 73820 (0.0009) -[2023-10-15 17:44:10,678][52866] Updated weights for policy 1, policy_version 74060 (0.0007) -[2023-10-15 17:44:11,042][52866] Updated weights for policy 1, policy_version 74070 (0.0008) -[2023-10-15 17:44:11,416][52866] Updated weights for policy 1, policy_version 74080 (0.0010) -[2023-10-15 17:44:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 151453696. Throughput: 0: 1790.4, 1: 1785.8. Samples: 37875824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:13,442][51532] Avg episode reward: [(0, '69.720'), (1, '65.400')] -[2023-10-15 17:44:14,080][52833] Updated weights for policy 0, policy_version 73830 (0.0008) -[2023-10-15 17:44:14,454][52833] Updated weights for policy 0, policy_version 73840 (0.0009) -[2023-10-15 17:44:14,818][52833] Updated weights for policy 0, policy_version 73850 (0.0009) -[2023-10-15 17:44:15,084][52866] Updated weights for policy 1, policy_version 74090 (0.0008) -[2023-10-15 17:44:15,439][52866] Updated weights for policy 1, policy_version 74100 (0.0010) -[2023-10-15 17:44:15,808][52866] Updated weights for policy 1, policy_version 74110 (0.0008) -[2023-10-15 17:44:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151519232. Throughput: 0: 1793.6, 1: 1782.2. Samples: 37885662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:18,441][51532] Avg episode reward: [(0, '73.040'), (1, '61.550')] -[2023-10-15 17:44:18,471][52833] Updated weights for policy 0, policy_version 73860 (0.0007) -[2023-10-15 17:44:18,835][52833] Updated weights for policy 0, policy_version 73870 (0.0007) -[2023-10-15 17:44:19,210][52833] Updated weights for policy 0, policy_version 73880 (0.0008) -[2023-10-15 17:44:19,494][52410] Saving new best policy, reward=73.040! -[2023-10-15 17:44:19,590][52866] Updated weights for policy 1, policy_version 74120 (0.0007) -[2023-10-15 17:44:19,954][52866] Updated weights for policy 1, policy_version 74130 (0.0007) -[2023-10-15 17:44:20,319][52866] Updated weights for policy 1, policy_version 74140 (0.0007) -[2023-10-15 17:44:22,801][52833] Updated weights for policy 0, policy_version 73890 (0.0009) -[2023-10-15 17:44:23,170][52833] Updated weights for policy 0, policy_version 73900 (0.0010) -[2023-10-15 17:44:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 151584768. Throughput: 0: 1801.1, 1: 1787.2. Samples: 37908494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:23,442][51532] Avg episode reward: [(0, '73.380'), (1, '63.470')] -[2023-10-15 17:44:23,549][52833] Updated weights for policy 0, policy_version 73910 (0.0008) -[2023-10-15 17:44:23,908][52410] Saving new best policy, reward=73.380! -[2023-10-15 17:44:23,912][52833] Updated weights for policy 0, policy_version 73920 (0.0009) -[2023-10-15 17:44:24,102][52866] Updated weights for policy 1, policy_version 74150 (0.0008) -[2023-10-15 17:44:24,465][52866] Updated weights for policy 1, policy_version 74160 (0.0008) -[2023-10-15 17:44:24,834][52866] Updated weights for policy 1, policy_version 74170 (0.0008) -[2023-10-15 17:44:27,573][52833] Updated weights for policy 0, policy_version 73930 (0.0008) -[2023-10-15 17:44:27,957][52833] Updated weights for policy 0, policy_version 73940 (0.0009) -[2023-10-15 17:44:28,326][52833] Updated weights for policy 0, policy_version 73950 (0.0009) -[2023-10-15 17:44:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 151683072. Throughput: 0: 1807.8, 1: 1793.6. Samples: 37930322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:28,442][51532] Avg episode reward: [(0, '75.570'), (1, '61.980')] -[2023-10-15 17:44:28,449][52410] Saving new best policy, reward=75.570! -[2023-10-15 17:44:28,581][52866] Updated weights for policy 1, policy_version 74180 (0.0008) -[2023-10-15 17:44:28,955][52866] Updated weights for policy 1, policy_version 74190 (0.0008) -[2023-10-15 17:44:29,323][52866] Updated weights for policy 1, policy_version 74200 (0.0011) -[2023-10-15 17:44:32,143][52833] Updated weights for policy 0, policy_version 73960 (0.0009) -[2023-10-15 17:44:32,513][52833] Updated weights for policy 0, policy_version 73970 (0.0007) -[2023-10-15 17:44:32,876][52833] Updated weights for policy 0, policy_version 73980 (0.0012) -[2023-10-15 17:44:33,045][52866] Updated weights for policy 1, policy_version 74210 (0.0010) -[2023-10-15 17:44:33,412][52866] Updated weights for policy 1, policy_version 74220 (0.0009) -[2023-10-15 17:44:33,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 151748608. Throughput: 0: 1802.3, 1: 1794.3. Samples: 37940974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:33,442][51532] Avg episode reward: [(0, '72.420'), (1, '61.960')] -[2023-10-15 17:44:33,770][52866] Updated weights for policy 1, policy_version 74230 (0.0010) -[2023-10-15 17:44:34,142][52866] Updated weights for policy 1, policy_version 74240 (0.0011) -[2023-10-15 17:44:36,475][52833] Updated weights for policy 0, policy_version 73990 (0.0009) -[2023-10-15 17:44:36,839][52833] Updated weights for policy 0, policy_version 74000 (0.0007) -[2023-10-15 17:44:37,204][52833] Updated weights for policy 0, policy_version 74010 (0.0008) -[2023-10-15 17:44:37,937][52866] Updated weights for policy 1, policy_version 74250 (0.0010) -[2023-10-15 17:44:38,296][52866] Updated weights for policy 1, policy_version 74260 (0.0008) -[2023-10-15 17:44:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 151814144. Throughput: 0: 1807.6, 1: 1795.4. Samples: 37962782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:38,441][51532] Avg episode reward: [(0, '72.910'), (1, '63.190')] -[2023-10-15 17:44:38,664][52866] Updated weights for policy 1, policy_version 74270 (0.0009) -[2023-10-15 17:44:41,243][52833] Updated weights for policy 0, policy_version 74020 (0.0009) -[2023-10-15 17:44:41,644][52833] Updated weights for policy 0, policy_version 74030 (0.0009) -[2023-10-15 17:44:42,029][52833] Updated weights for policy 0, policy_version 74040 (0.0008) -[2023-10-15 17:44:42,451][52866] Updated weights for policy 1, policy_version 74280 (0.0010) -[2023-10-15 17:44:42,818][52866] Updated weights for policy 1, policy_version 74290 (0.0008) -[2023-10-15 17:44:43,187][52866] Updated weights for policy 1, policy_version 74300 (0.0008) -[2023-10-15 17:44:43,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 151912448. Throughput: 0: 1797.5, 1: 1801.6. Samples: 37983096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:43,442][51532] Avg episode reward: [(0, '69.670'), (1, '64.230')] -[2023-10-15 17:44:45,823][52833] Updated weights for policy 0, policy_version 74050 (0.0008) -[2023-10-15 17:44:46,186][52833] Updated weights for policy 0, policy_version 74060 (0.0009) -[2023-10-15 17:44:46,552][52833] Updated weights for policy 0, policy_version 74070 (0.0010) -[2023-10-15 17:44:46,921][52833] Updated weights for policy 0, policy_version 74080 (0.0008) -[2023-10-15 17:44:46,925][52866] Updated weights for policy 1, policy_version 74310 (0.0008) -[2023-10-15 17:44:47,299][52866] Updated weights for policy 1, policy_version 74320 (0.0007) -[2023-10-15 17:44:47,670][52866] Updated weights for policy 1, policy_version 74330 (0.0007) -[2023-10-15 17:44:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 151977984. Throughput: 0: 1808.2, 1: 1791.3. Samples: 37995124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:48,442][51532] Avg episode reward: [(0, '67.950'), (1, '63.110')] -[2023-10-15 17:44:50,685][52833] Updated weights for policy 0, policy_version 74090 (0.0009) -[2023-10-15 17:44:51,050][52833] Updated weights for policy 0, policy_version 74100 (0.0010) -[2023-10-15 17:44:51,381][52866] Updated weights for policy 1, policy_version 74340 (0.0009) -[2023-10-15 17:44:51,428][52833] Updated weights for policy 0, policy_version 74110 (0.0008) -[2023-10-15 17:44:51,745][52866] Updated weights for policy 1, policy_version 74350 (0.0007) -[2023-10-15 17:44:52,110][52866] Updated weights for policy 1, policy_version 74360 (0.0007) -[2023-10-15 17:44:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152043520. Throughput: 0: 1797.1, 1: 1802.7. Samples: 38015284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:53,441][51532] Avg episode reward: [(0, '70.810'), (1, '63.370')] -[2023-10-15 17:44:55,382][52833] Updated weights for policy 0, policy_version 74120 (0.0012) -[2023-10-15 17:44:55,754][52833] Updated weights for policy 0, policy_version 74130 (0.0009) -[2023-10-15 17:44:55,782][52866] Updated weights for policy 1, policy_version 74370 (0.0009) -[2023-10-15 17:44:56,124][52833] Updated weights for policy 0, policy_version 74140 (0.0009) -[2023-10-15 17:44:56,143][52866] Updated weights for policy 1, policy_version 74380 (0.0009) -[2023-10-15 17:44:56,508][52866] Updated weights for policy 1, policy_version 74390 (0.0010) -[2023-10-15 17:44:56,876][52866] Updated weights for policy 1, policy_version 74400 (0.0011) -[2023-10-15 17:44:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152109056. Throughput: 0: 1787.8, 1: 1800.6. Samples: 38037300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:44:58,442][51532] Avg episode reward: [(0, '67.690'), (1, '58.970')] -[2023-10-15 17:44:59,989][52833] Updated weights for policy 0, policy_version 74150 (0.0008) -[2023-10-15 17:45:00,356][52833] Updated weights for policy 0, policy_version 74160 (0.0008) -[2023-10-15 17:45:00,651][52866] Updated weights for policy 1, policy_version 74410 (0.0007) -[2023-10-15 17:45:00,729][52833] Updated weights for policy 0, policy_version 74170 (0.0008) -[2023-10-15 17:45:01,013][52866] Updated weights for policy 1, policy_version 74420 (0.0007) -[2023-10-15 17:45:01,387][52866] Updated weights for policy 1, policy_version 74430 (0.0009) -[2023-10-15 17:45:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 152174592. Throughput: 0: 1787.1, 1: 1816.4. Samples: 38047822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:03,442][51532] Avg episode reward: [(0, '69.230'), (1, '57.390')] -[2023-10-15 17:45:04,382][52833] Updated weights for policy 0, policy_version 74180 (0.0007) -[2023-10-15 17:45:04,760][52833] Updated weights for policy 0, policy_version 74190 (0.0009) -[2023-10-15 17:45:05,038][52866] Updated weights for policy 1, policy_version 74440 (0.0008) -[2023-10-15 17:45:05,127][52833] Updated weights for policy 0, policy_version 74200 (0.0009) -[2023-10-15 17:45:05,400][52866] Updated weights for policy 1, policy_version 74450 (0.0009) -[2023-10-15 17:45:05,773][52866] Updated weights for policy 1, policy_version 74460 (0.0008) -[2023-10-15 17:45:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152240128. Throughput: 0: 1776.4, 1: 1803.2. Samples: 38069576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:08,442][51532] Avg episode reward: [(0, '67.100'), (1, '58.790')] -[2023-10-15 17:45:08,931][52833] Updated weights for policy 0, policy_version 74210 (0.0008) -[2023-10-15 17:45:09,301][52833] Updated weights for policy 0, policy_version 74220 (0.0009) -[2023-10-15 17:45:09,494][52866] Updated weights for policy 1, policy_version 74470 (0.0009) -[2023-10-15 17:45:09,659][52833] Updated weights for policy 0, policy_version 74230 (0.0008) -[2023-10-15 17:45:09,860][52866] Updated weights for policy 1, policy_version 74480 (0.0008) -[2023-10-15 17:45:10,034][52833] Updated weights for policy 0, policy_version 74240 (0.0009) -[2023-10-15 17:45:10,224][52866] Updated weights for policy 1, policy_version 74490 (0.0008) -[2023-10-15 17:45:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 152305664. Throughput: 0: 1793.4, 1: 1799.0. Samples: 38091980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:13,442][51532] Avg episode reward: [(0, '66.910'), (1, '61.590')] -[2023-10-15 17:45:13,731][52833] Updated weights for policy 0, policy_version 74250 (0.0007) -[2023-10-15 17:45:14,022][52866] Updated weights for policy 1, policy_version 74500 (0.0008) -[2023-10-15 17:45:14,098][52833] Updated weights for policy 0, policy_version 74260 (0.0008) -[2023-10-15 17:45:14,381][52866] Updated weights for policy 1, policy_version 74510 (0.0009) -[2023-10-15 17:45:14,473][52833] Updated weights for policy 0, policy_version 74270 (0.0008) -[2023-10-15 17:45:14,749][52866] Updated weights for policy 1, policy_version 74520 (0.0007) -[2023-10-15 17:45:18,168][52833] Updated weights for policy 0, policy_version 74280 (0.0010) -[2023-10-15 17:45:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 152371200. Throughput: 0: 1773.2, 1: 1796.4. Samples: 38101606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:18,441][51532] Avg episode reward: [(0, '67.070'), (1, '60.040')] -[2023-10-15 17:45:18,540][52833] Updated weights for policy 0, policy_version 74290 (0.0010) -[2023-10-15 17:45:18,563][52866] Updated weights for policy 1, policy_version 74530 (0.0008) -[2023-10-15 17:45:18,914][52833] Updated weights for policy 0, policy_version 74300 (0.0007) -[2023-10-15 17:45:18,928][52866] Updated weights for policy 1, policy_version 74540 (0.0007) -[2023-10-15 17:45:19,302][52866] Updated weights for policy 1, policy_version 74550 (0.0008) -[2023-10-15 17:45:19,671][52866] Updated weights for policy 1, policy_version 74560 (0.0009) -[2023-10-15 17:45:22,691][52833] Updated weights for policy 0, policy_version 74310 (0.0009) -[2023-10-15 17:45:23,065][52833] Updated weights for policy 0, policy_version 74320 (0.0010) -[2023-10-15 17:45:23,392][52866] Updated weights for policy 1, policy_version 74570 (0.0007) -[2023-10-15 17:45:23,432][52833] Updated weights for policy 0, policy_version 74330 (0.0009) -[2023-10-15 17:45:23,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 152436736. Throughput: 0: 1786.8, 1: 1794.8. Samples: 38123958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:23,441][51532] Avg episode reward: [(0, '67.030'), (1, '57.880')] -[2023-10-15 17:45:23,754][52866] Updated weights for policy 1, policy_version 74580 (0.0009) -[2023-10-15 17:45:24,124][52866] Updated weights for policy 1, policy_version 74590 (0.0010) -[2023-10-15 17:45:27,329][52833] Updated weights for policy 0, policy_version 74340 (0.0008) -[2023-10-15 17:45:27,722][52833] Updated weights for policy 0, policy_version 74350 (0.0008) -[2023-10-15 17:45:27,828][52866] Updated weights for policy 1, policy_version 74600 (0.0008) -[2023-10-15 17:45:28,095][52833] Updated weights for policy 0, policy_version 74360 (0.0008) -[2023-10-15 17:45:28,193][52866] Updated weights for policy 1, policy_version 74610 (0.0007) -[2023-10-15 17:45:28,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 152535040. Throughput: 0: 1788.7, 1: 1808.0. Samples: 38144946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:28,442][51532] Avg episode reward: [(0, '67.400'), (1, '53.870')] -[2023-10-15 17:45:28,555][52866] Updated weights for policy 1, policy_version 74620 (0.0008) -[2023-10-15 17:45:31,739][52833] Updated weights for policy 0, policy_version 74370 (0.0008) -[2023-10-15 17:45:32,111][52833] Updated weights for policy 0, policy_version 74380 (0.0007) -[2023-10-15 17:45:32,438][52866] Updated weights for policy 1, policy_version 74630 (0.0007) -[2023-10-15 17:45:32,476][52833] Updated weights for policy 0, policy_version 74390 (0.0007) -[2023-10-15 17:45:32,814][52866] Updated weights for policy 1, policy_version 74640 (0.0008) -[2023-10-15 17:45:32,843][52833] Updated weights for policy 0, policy_version 74400 (0.0010) -[2023-10-15 17:45:33,183][52866] Updated weights for policy 1, policy_version 74650 (0.0007) -[2023-10-15 17:45:33,441][51532] Fps is (10 sec: 19660.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 152633344. Throughput: 0: 1778.0, 1: 1795.1. Samples: 38155916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:33,442][51532] Avg episode reward: [(0, '70.560'), (1, '50.460')] -[2023-10-15 17:45:36,613][52833] Updated weights for policy 0, policy_version 74410 (0.0009) -[2023-10-15 17:45:36,981][52833] Updated weights for policy 0, policy_version 74420 (0.0009) -[2023-10-15 17:45:37,057][52866] Updated weights for policy 1, policy_version 74660 (0.0008) -[2023-10-15 17:45:37,347][52833] Updated weights for policy 0, policy_version 74430 (0.0007) -[2023-10-15 17:45:37,416][52866] Updated weights for policy 1, policy_version 74670 (0.0009) -[2023-10-15 17:45:37,788][52866] Updated weights for policy 1, policy_version 74680 (0.0007) -[2023-10-15 17:45:38,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 152698880. Throughput: 0: 1785.1, 1: 1810.4. Samples: 38177080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:38,442][51532] Avg episode reward: [(0, '68.470'), (1, '52.010')] -[2023-10-15 17:45:41,185][52833] Updated weights for policy 0, policy_version 74440 (0.0008) -[2023-10-15 17:45:41,499][52866] Updated weights for policy 1, policy_version 74690 (0.0007) -[2023-10-15 17:45:41,555][52833] Updated weights for policy 0, policy_version 74450 (0.0008) -[2023-10-15 17:45:41,866][52866] Updated weights for policy 1, policy_version 74700 (0.0008) -[2023-10-15 17:45:41,929][52833] Updated weights for policy 0, policy_version 74460 (0.0007) -[2023-10-15 17:45:42,227][52866] Updated weights for policy 1, policy_version 74710 (0.0008) -[2023-10-15 17:45:42,595][52866] Updated weights for policy 1, policy_version 74720 (0.0009) -[2023-10-15 17:45:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152764416. Throughput: 0: 1772.7, 1: 1784.3. Samples: 38197366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:43,441][51532] Avg episode reward: [(0, '69.270'), (1, '51.950')] -[2023-10-15 17:45:43,449][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000074720_76513280.pth... -[2023-10-15 17:45:43,449][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000074464_76251136.pth... -[2023-10-15 17:45:43,486][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000072768_74514432.pth -[2023-10-15 17:45:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000073024_74776576.pth -[2023-10-15 17:45:45,756][52833] Updated weights for policy 0, policy_version 74470 (0.0007) -[2023-10-15 17:45:46,117][52833] Updated weights for policy 0, policy_version 74480 (0.0008) -[2023-10-15 17:45:46,254][52866] Updated weights for policy 1, policy_version 74730 (0.0008) -[2023-10-15 17:45:46,483][52833] Updated weights for policy 0, policy_version 74490 (0.0007) -[2023-10-15 17:45:46,614][52866] Updated weights for policy 1, policy_version 74740 (0.0008) -[2023-10-15 17:45:46,980][52866] Updated weights for policy 1, policy_version 74750 (0.0008) -[2023-10-15 17:45:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152829952. Throughput: 0: 1792.1, 1: 1799.9. Samples: 38209464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:48,441][51532] Avg episode reward: [(0, '68.730'), (1, '51.200')] -[2023-10-15 17:45:50,212][52833] Updated weights for policy 0, policy_version 74500 (0.0008) -[2023-10-15 17:45:50,584][52833] Updated weights for policy 0, policy_version 74510 (0.0009) -[2023-10-15 17:45:50,698][52866] Updated weights for policy 1, policy_version 74760 (0.0008) -[2023-10-15 17:45:50,946][52833] Updated weights for policy 0, policy_version 74520 (0.0007) -[2023-10-15 17:45:51,064][52866] Updated weights for policy 1, policy_version 74770 (0.0009) -[2023-10-15 17:45:51,421][52866] Updated weights for policy 1, policy_version 74780 (0.0009) -[2023-10-15 17:45:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152895488. Throughput: 0: 1777.8, 1: 1777.3. Samples: 38229558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:45:53,442][51532] Avg episode reward: [(0, '68.990'), (1, '53.050')] -[2023-10-15 17:45:54,701][52833] Updated weights for policy 0, policy_version 74530 (0.0009) -[2023-10-15 17:45:55,068][52833] Updated weights for policy 0, policy_version 74540 (0.0008) -[2023-10-15 17:45:55,218][52866] Updated weights for policy 1, policy_version 74790 (0.0009) -[2023-10-15 17:45:55,439][52833] Updated weights for policy 0, policy_version 74550 (0.0008) -[2023-10-15 17:45:55,580][52866] Updated weights for policy 1, policy_version 74800 (0.0008) -[2023-10-15 17:45:55,798][52833] Updated weights for policy 0, policy_version 74560 (0.0008) -[2023-10-15 17:45:55,952][52866] Updated weights for policy 1, policy_version 74810 (0.0007) -[2023-10-15 17:45:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152961024. Throughput: 0: 1776.2, 1: 1780.1. Samples: 38252010. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:45:58,441][51532] Avg episode reward: [(0, '70.780'), (1, '48.900')] -[2023-10-15 17:45:59,455][52833] Updated weights for policy 0, policy_version 74570 (0.0007) -[2023-10-15 17:45:59,797][52866] Updated weights for policy 1, policy_version 74820 (0.0008) -[2023-10-15 17:45:59,831][52833] Updated weights for policy 0, policy_version 74580 (0.0010) -[2023-10-15 17:46:00,163][52866] Updated weights for policy 1, policy_version 74830 (0.0008) -[2023-10-15 17:46:00,198][52833] Updated weights for policy 0, policy_version 74590 (0.0008) -[2023-10-15 17:46:00,528][52866] Updated weights for policy 1, policy_version 74840 (0.0008) -[2023-10-15 17:46:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 153026560. Throughput: 0: 1780.7, 1: 1781.7. Samples: 38261914. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:03,442][51532] Avg episode reward: [(0, '72.420'), (1, '44.770')] -[2023-10-15 17:46:03,968][52833] Updated weights for policy 0, policy_version 74600 (0.0009) -[2023-10-15 17:46:04,329][52833] Updated weights for policy 0, policy_version 74610 (0.0009) -[2023-10-15 17:46:04,360][52866] Updated weights for policy 1, policy_version 74850 (0.0007) -[2023-10-15 17:46:04,695][52833] Updated weights for policy 0, policy_version 74620 (0.0007) -[2023-10-15 17:46:04,727][52866] Updated weights for policy 1, policy_version 74860 (0.0008) -[2023-10-15 17:46:05,095][52866] Updated weights for policy 1, policy_version 74870 (0.0008) -[2023-10-15 17:46:05,454][52866] Updated weights for policy 1, policy_version 74880 (0.0007) -[2023-10-15 17:46:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 153092096. Throughput: 0: 1783.5, 1: 1779.6. Samples: 38284296. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:08,442][51532] Avg episode reward: [(0, '72.560'), (1, '48.870')] -[2023-10-15 17:46:08,560][52833] Updated weights for policy 0, policy_version 74630 (0.0009) -[2023-10-15 17:46:08,933][52833] Updated weights for policy 0, policy_version 74640 (0.0008) -[2023-10-15 17:46:09,178][52866] Updated weights for policy 1, policy_version 74890 (0.0007) -[2023-10-15 17:46:09,308][52833] Updated weights for policy 0, policy_version 74650 (0.0007) -[2023-10-15 17:46:09,538][52866] Updated weights for policy 1, policy_version 74900 (0.0008) -[2023-10-15 17:46:09,910][52866] Updated weights for policy 1, policy_version 74910 (0.0011) -[2023-10-15 17:46:13,170][52833] Updated weights for policy 0, policy_version 74660 (0.0009) -[2023-10-15 17:46:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 153157632. Throughput: 0: 1807.0, 1: 1791.7. Samples: 38306888. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:13,441][51532] Avg episode reward: [(0, '72.930'), (1, '49.490')] -[2023-10-15 17:46:13,567][52833] Updated weights for policy 0, policy_version 74670 (0.0007) -[2023-10-15 17:46:13,613][52866] Updated weights for policy 1, policy_version 74920 (0.0009) -[2023-10-15 17:46:13,933][52833] Updated weights for policy 0, policy_version 74680 (0.0007) -[2023-10-15 17:46:13,981][52866] Updated weights for policy 1, policy_version 74930 (0.0008) -[2023-10-15 17:46:14,336][52866] Updated weights for policy 1, policy_version 74940 (0.0008) -[2023-10-15 17:46:17,583][52833] Updated weights for policy 0, policy_version 74690 (0.0008) -[2023-10-15 17:46:17,941][52833] Updated weights for policy 0, policy_version 74700 (0.0007) -[2023-10-15 17:46:18,220][52866] Updated weights for policy 1, policy_version 74950 (0.0010) -[2023-10-15 17:46:18,315][52833] Updated weights for policy 0, policy_version 74710 (0.0007) -[2023-10-15 17:46:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 153223168. Throughput: 0: 1784.7, 1: 1782.2. Samples: 38316428. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:18,442][51532] Avg episode reward: [(0, '70.970'), (1, '53.670')] -[2023-10-15 17:46:18,582][52866] Updated weights for policy 1, policy_version 74960 (0.0008) -[2023-10-15 17:46:18,671][52833] Updated weights for policy 0, policy_version 74720 (0.0007) -[2023-10-15 17:46:18,946][52866] Updated weights for policy 1, policy_version 74970 (0.0010) -[2023-10-15 17:46:22,499][52833] Updated weights for policy 0, policy_version 74730 (0.0010) -[2023-10-15 17:46:22,685][52866] Updated weights for policy 1, policy_version 74980 (0.0009) -[2023-10-15 17:46:22,877][52833] Updated weights for policy 0, policy_version 74740 (0.0009) -[2023-10-15 17:46:23,048][52866] Updated weights for policy 1, policy_version 74990 (0.0008) -[2023-10-15 17:46:23,233][52833] Updated weights for policy 0, policy_version 74750 (0.0009) -[2023-10-15 17:46:23,413][52866] Updated weights for policy 1, policy_version 75000 (0.0007) -[2023-10-15 17:46:23,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 153321472. Throughput: 0: 1807.2, 1: 1787.6. Samples: 38338850. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:23,442][51532] Avg episode reward: [(0, '73.890'), (1, '53.350')] -[2023-10-15 17:46:26,959][52833] Updated weights for policy 0, policy_version 74760 (0.0009) -[2023-10-15 17:46:27,337][52833] Updated weights for policy 0, policy_version 74770 (0.0009) -[2023-10-15 17:46:27,360][52866] Updated weights for policy 1, policy_version 75010 (0.0007) -[2023-10-15 17:46:27,710][52833] Updated weights for policy 0, policy_version 74780 (0.0007) -[2023-10-15 17:46:27,718][52866] Updated weights for policy 1, policy_version 75020 (0.0008) -[2023-10-15 17:46:28,092][52866] Updated weights for policy 1, policy_version 75030 (0.0008) -[2023-10-15 17:46:28,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 153387008. Throughput: 0: 1790.8, 1: 1798.3. Samples: 38358878. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:28,441][51532] Avg episode reward: [(0, '70.040'), (1, '56.220')] -[2023-10-15 17:46:28,451][52866] Updated weights for policy 1, policy_version 75040 (0.0007) -[2023-10-15 17:46:31,187][52833] Updated weights for policy 0, policy_version 74790 (0.0009) -[2023-10-15 17:46:31,564][52833] Updated weights for policy 0, policy_version 74800 (0.0011) -[2023-10-15 17:46:31,930][52833] Updated weights for policy 0, policy_version 74810 (0.0011) -[2023-10-15 17:46:32,257][52866] Updated weights for policy 1, policy_version 75050 (0.0009) -[2023-10-15 17:46:32,615][52866] Updated weights for policy 1, policy_version 75060 (0.0010) -[2023-10-15 17:46:32,992][52866] Updated weights for policy 1, policy_version 75070 (0.0008) -[2023-10-15 17:46:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 153485312. Throughput: 0: 1806.3, 1: 1785.7. Samples: 38371108. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:33,442][51532] Avg episode reward: [(0, '69.710'), (1, '56.850')] -[2023-10-15 17:46:35,739][52833] Updated weights for policy 0, policy_version 74820 (0.0009) -[2023-10-15 17:46:36,108][52833] Updated weights for policy 0, policy_version 74830 (0.0010) -[2023-10-15 17:46:36,483][52833] Updated weights for policy 0, policy_version 74840 (0.0009) -[2023-10-15 17:46:36,688][52866] Updated weights for policy 1, policy_version 75080 (0.0009) -[2023-10-15 17:46:37,050][52866] Updated weights for policy 1, policy_version 75090 (0.0010) -[2023-10-15 17:46:37,416][52866] Updated weights for policy 1, policy_version 75100 (0.0009) -[2023-10-15 17:46:38,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 153550848. Throughput: 0: 1791.0, 1: 1804.8. Samples: 38391370. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:38,442][51532] Avg episode reward: [(0, '70.540'), (1, '55.560')] -[2023-10-15 17:46:40,357][52833] Updated weights for policy 0, policy_version 74850 (0.0007) -[2023-10-15 17:46:40,722][52833] Updated weights for policy 0, policy_version 74860 (0.0007) -[2023-10-15 17:46:41,091][52833] Updated weights for policy 0, policy_version 74870 (0.0008) -[2023-10-15 17:46:41,181][52866] Updated weights for policy 1, policy_version 75110 (0.0008) -[2023-10-15 17:46:41,453][52833] Updated weights for policy 0, policy_version 74880 (0.0008) -[2023-10-15 17:46:41,544][52866] Updated weights for policy 1, policy_version 75120 (0.0009) -[2023-10-15 17:46:41,907][52866] Updated weights for policy 1, policy_version 75130 (0.0007) -[2023-10-15 17:46:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 153616384. Throughput: 0: 1793.2, 1: 1791.5. Samples: 38413320. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-15 17:46:43,441][51532] Avg episode reward: [(0, '73.920'), (1, '59.180')] -[2023-10-15 17:46:45,104][52833] Updated weights for policy 0, policy_version 74890 (0.0007) -[2023-10-15 17:46:45,476][52833] Updated weights for policy 0, policy_version 74900 (0.0007) -[2023-10-15 17:46:45,492][52866] Updated weights for policy 1, policy_version 75140 (0.0007) -[2023-10-15 17:46:45,846][52833] Updated weights for policy 0, policy_version 74910 (0.0007) -[2023-10-15 17:46:45,861][52866] Updated weights for policy 1, policy_version 75150 (0.0007) -[2023-10-15 17:46:46,221][52866] Updated weights for policy 1, policy_version 75160 (0.0009) -[2023-10-15 17:46:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 153681920. Throughput: 0: 1796.2, 1: 1805.9. Samples: 38424010. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:46:48,441][51532] Avg episode reward: [(0, '75.000'), (1, '59.110')] -[2023-10-15 17:46:49,625][52833] Updated weights for policy 0, policy_version 74920 (0.0007) -[2023-10-15 17:46:49,999][52833] Updated weights for policy 0, policy_version 74930 (0.0007) -[2023-10-15 17:46:50,079][52866] Updated weights for policy 1, policy_version 75170 (0.0010) -[2023-10-15 17:46:50,374][52833] Updated weights for policy 0, policy_version 74940 (0.0007) -[2023-10-15 17:46:50,443][52866] Updated weights for policy 1, policy_version 75180 (0.0007) -[2023-10-15 17:46:50,811][52866] Updated weights for policy 1, policy_version 75190 (0.0010) -[2023-10-15 17:46:51,166][52866] Updated weights for policy 1, policy_version 75200 (0.0009) -[2023-10-15 17:46:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 153747456. Throughput: 0: 1788.2, 1: 1791.2. Samples: 38445372. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:46:53,442][51532] Avg episode reward: [(0, '75.590'), (1, '58.600')] -[2023-10-15 17:46:53,442][52410] Saving new best policy, reward=75.590! -[2023-10-15 17:46:54,170][52833] Updated weights for policy 0, policy_version 74950 (0.0008) -[2023-10-15 17:46:54,531][52833] Updated weights for policy 0, policy_version 74960 (0.0007) -[2023-10-15 17:46:54,901][52833] Updated weights for policy 0, policy_version 74970 (0.0008) -[2023-10-15 17:46:54,908][52866] Updated weights for policy 1, policy_version 75210 (0.0009) -[2023-10-15 17:46:55,281][52866] Updated weights for policy 1, policy_version 75220 (0.0009) -[2023-10-15 17:46:55,642][52866] Updated weights for policy 1, policy_version 75230 (0.0007) -[2023-10-15 17:46:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 153812992. Throughput: 0: 1790.1, 1: 1790.0. Samples: 38467992. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:46:58,442][51532] Avg episode reward: [(0, '75.460'), (1, '61.420')] -[2023-10-15 17:46:58,641][52833] Updated weights for policy 0, policy_version 74980 (0.0007) -[2023-10-15 17:46:59,026][52833] Updated weights for policy 0, policy_version 74990 (0.0008) -[2023-10-15 17:46:59,299][52866] Updated weights for policy 1, policy_version 75240 (0.0007) -[2023-10-15 17:46:59,397][52833] Updated weights for policy 0, policy_version 75000 (0.0008) -[2023-10-15 17:46:59,665][52866] Updated weights for policy 1, policy_version 75250 (0.0008) -[2023-10-15 17:47:00,027][52866] Updated weights for policy 1, policy_version 75260 (0.0007) -[2023-10-15 17:47:02,976][52833] Updated weights for policy 0, policy_version 75010 (0.0007) -[2023-10-15 17:47:03,357][52833] Updated weights for policy 0, policy_version 75020 (0.0008) -[2023-10-15 17:47:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 153878528. Throughput: 0: 1795.2, 1: 1794.4. Samples: 38477962. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:47:03,441][51532] Avg episode reward: [(0, '74.040'), (1, '62.110')] -[2023-10-15 17:47:03,724][52833] Updated weights for policy 0, policy_version 75030 (0.0009) -[2023-10-15 17:47:03,796][52866] Updated weights for policy 1, policy_version 75270 (0.0007) -[2023-10-15 17:47:04,092][52833] Updated weights for policy 0, policy_version 75040 (0.0008) -[2023-10-15 17:47:04,177][52866] Updated weights for policy 1, policy_version 75280 (0.0009) -[2023-10-15 17:47:04,546][52866] Updated weights for policy 1, policy_version 75290 (0.0008) -[2023-10-15 17:47:07,893][52833] Updated weights for policy 0, policy_version 75050 (0.0007) -[2023-10-15 17:47:08,209][52866] Updated weights for policy 1, policy_version 75300 (0.0008) -[2023-10-15 17:47:08,261][52833] Updated weights for policy 0, policy_version 75060 (0.0010) -[2023-10-15 17:47:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 153944064. Throughput: 0: 1793.7, 1: 1794.7. Samples: 38500330. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:47:08,442][51532] Avg episode reward: [(0, '74.290'), (1, '61.990')] -[2023-10-15 17:47:08,574][52866] Updated weights for policy 1, policy_version 75310 (0.0009) -[2023-10-15 17:47:08,640][52833] Updated weights for policy 0, policy_version 75070 (0.0009) -[2023-10-15 17:47:08,944][52866] Updated weights for policy 1, policy_version 75320 (0.0008) -[2023-10-15 17:47:12,469][52833] Updated weights for policy 0, policy_version 75080 (0.0007) -[2023-10-15 17:47:12,761][52866] Updated weights for policy 1, policy_version 75330 (0.0010) -[2023-10-15 17:47:12,841][52833] Updated weights for policy 0, policy_version 75090 (0.0007) -[2023-10-15 17:47:13,124][52866] Updated weights for policy 1, policy_version 75340 (0.0007) -[2023-10-15 17:47:13,207][52833] Updated weights for policy 0, policy_version 75100 (0.0008) -[2023-10-15 17:47:13,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 154042368. Throughput: 0: 1805.1, 1: 1813.0. Samples: 38521694. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:47:13,442][51532] Avg episode reward: [(0, '74.570'), (1, '59.800')] -[2023-10-15 17:47:13,484][52866] Updated weights for policy 1, policy_version 75350 (0.0008) -[2023-10-15 17:47:13,850][52866] Updated weights for policy 1, policy_version 75360 (0.0008) -[2023-10-15 17:47:17,004][52833] Updated weights for policy 0, policy_version 75110 (0.0007) -[2023-10-15 17:47:17,367][52833] Updated weights for policy 0, policy_version 75120 (0.0007) -[2023-10-15 17:47:17,576][52866] Updated weights for policy 1, policy_version 75370 (0.0008) -[2023-10-15 17:47:17,731][52833] Updated weights for policy 0, policy_version 75130 (0.0008) -[2023-10-15 17:47:17,944][52866] Updated weights for policy 1, policy_version 75380 (0.0007) -[2023-10-15 17:47:18,320][52866] Updated weights for policy 1, policy_version 75390 (0.0009) -[2023-10-15 17:47:18,441][51532] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 154140672. Throughput: 0: 1786.0, 1: 1800.0. Samples: 38532476. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:47:18,442][51532] Avg episode reward: [(0, '71.160'), (1, '59.000')] -[2023-10-15 17:47:21,567][52833] Updated weights for policy 0, policy_version 75140 (0.0009) -[2023-10-15 17:47:21,934][52833] Updated weights for policy 0, policy_version 75150 (0.0008) -[2023-10-15 17:47:22,237][52866] Updated weights for policy 1, policy_version 75400 (0.0008) -[2023-10-15 17:47:22,301][52833] Updated weights for policy 0, policy_version 75160 (0.0007) -[2023-10-15 17:47:22,608][52866] Updated weights for policy 1, policy_version 75410 (0.0007) -[2023-10-15 17:47:22,969][52866] Updated weights for policy 1, policy_version 75420 (0.0008) -[2023-10-15 17:47:23,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14440.2). Total num frames: 154206208. Throughput: 0: 1808.5, 1: 1808.0. Samples: 38554112. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:47:23,441][51532] Avg episode reward: [(0, '69.540'), (1, '61.270')] -[2023-10-15 17:47:26,098][52833] Updated weights for policy 0, policy_version 75170 (0.0008) -[2023-10-15 17:47:26,469][52833] Updated weights for policy 0, policy_version 75180 (0.0008) -[2023-10-15 17:47:26,836][52833] Updated weights for policy 0, policy_version 75190 (0.0009) -[2023-10-15 17:47:26,873][52866] Updated weights for policy 1, policy_version 75430 (0.0009) -[2023-10-15 17:47:27,207][52833] Updated weights for policy 0, policy_version 75200 (0.0010) -[2023-10-15 17:47:27,243][52866] Updated weights for policy 1, policy_version 75440 (0.0007) -[2023-10-15 17:47:27,615][52866] Updated weights for policy 1, policy_version 75450 (0.0011) -[2023-10-15 17:47:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 154271744. Throughput: 0: 1787.5, 1: 1784.4. Samples: 38574056. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:47:28,441][51532] Avg episode reward: [(0, '70.670'), (1, '62.330')] -[2023-10-15 17:47:30,840][52833] Updated weights for policy 0, policy_version 75210 (0.0010) -[2023-10-15 17:47:31,209][52833] Updated weights for policy 0, policy_version 75220 (0.0008) -[2023-10-15 17:47:31,338][52866] Updated weights for policy 1, policy_version 75460 (0.0009) -[2023-10-15 17:47:31,568][52833] Updated weights for policy 0, policy_version 75230 (0.0008) -[2023-10-15 17:47:31,699][52866] Updated weights for policy 1, policy_version 75470 (0.0007) -[2023-10-15 17:47:32,068][52866] Updated weights for policy 1, policy_version 75480 (0.0007) -[2023-10-15 17:47:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154337280. Throughput: 0: 1807.2, 1: 1802.7. Samples: 38586456. Policy #0 lag: (min: 25.0, avg: 31.6, max: 57.0) -[2023-10-15 17:47:33,441][51532] Avg episode reward: [(0, '70.270'), (1, '63.450')] -[2023-10-15 17:47:35,266][52833] Updated weights for policy 0, policy_version 75240 (0.0008) -[2023-10-15 17:47:35,637][52833] Updated weights for policy 0, policy_version 75250 (0.0009) -[2023-10-15 17:47:35,858][52866] Updated weights for policy 1, policy_version 75490 (0.0009) -[2023-10-15 17:47:36,002][52833] Updated weights for policy 0, policy_version 75260 (0.0009) -[2023-10-15 17:47:36,220][52866] Updated weights for policy 1, policy_version 75500 (0.0009) -[2023-10-15 17:47:36,582][52866] Updated weights for policy 1, policy_version 75510 (0.0008) -[2023-10-15 17:47:36,944][52866] Updated weights for policy 1, policy_version 75520 (0.0007) -[2023-10-15 17:47:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154402816. Throughput: 0: 1792.5, 1: 1788.3. Samples: 38606508. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:47:38,441][51532] Avg episode reward: [(0, '64.940'), (1, '60.750')] -[2023-10-15 17:47:39,709][52833] Updated weights for policy 0, policy_version 75270 (0.0009) -[2023-10-15 17:47:40,074][52833] Updated weights for policy 0, policy_version 75280 (0.0007) -[2023-10-15 17:47:40,452][52833] Updated weights for policy 0, policy_version 75290 (0.0009) -[2023-10-15 17:47:40,668][52866] Updated weights for policy 1, policy_version 75530 (0.0009) -[2023-10-15 17:47:41,031][52866] Updated weights for policy 1, policy_version 75540 (0.0010) -[2023-10-15 17:47:41,404][52866] Updated weights for policy 1, policy_version 75550 (0.0010) -[2023-10-15 17:47:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 154468352. Throughput: 0: 1795.5, 1: 1779.1. Samples: 38628852. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:47:43,442][51532] Avg episode reward: [(0, '65.690'), (1, '61.200')] -[2023-10-15 17:47:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000075296_77103104.pth... -[2023-10-15 17:47:43,453][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000075552_77365248.pth... -[2023-10-15 17:47:43,489][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000073888_75661312.pth -[2023-10-15 17:47:43,493][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000075552_77365248.pth -[2023-10-15 17:47:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000073632_75399168.pth -[2023-10-15 17:47:43,499][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000075296_77103104.pth -[2023-10-15 17:47:44,343][52833] Updated weights for policy 0, policy_version 75300 (0.0008) -[2023-10-15 17:47:44,710][52833] Updated weights for policy 0, policy_version 75310 (0.0008) -[2023-10-15 17:47:45,070][52833] Updated weights for policy 0, policy_version 75320 (0.0010) -[2023-10-15 17:47:45,284][52866] Updated weights for policy 1, policy_version 75560 (0.0008) -[2023-10-15 17:47:45,651][52866] Updated weights for policy 1, policy_version 75570 (0.0007) -[2023-10-15 17:47:46,007][52866] Updated weights for policy 1, policy_version 75580 (0.0009) -[2023-10-15 17:47:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 154533888. Throughput: 0: 1794.9, 1: 1778.3. Samples: 38638756. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:47:48,441][51532] Avg episode reward: [(0, '66.560'), (1, '60.860')] -[2023-10-15 17:47:48,794][52833] Updated weights for policy 0, policy_version 75330 (0.0008) -[2023-10-15 17:47:49,161][52833] Updated weights for policy 0, policy_version 75340 (0.0008) -[2023-10-15 17:47:49,523][52833] Updated weights for policy 0, policy_version 75350 (0.0008) -[2023-10-15 17:47:49,889][52866] Updated weights for policy 1, policy_version 75590 (0.0008) -[2023-10-15 17:47:49,893][52833] Updated weights for policy 0, policy_version 75360 (0.0008) -[2023-10-15 17:47:50,248][52866] Updated weights for policy 1, policy_version 75600 (0.0009) -[2023-10-15 17:47:50,617][52866] Updated weights for policy 1, policy_version 75610 (0.0011) -[2023-10-15 17:47:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 154599424. Throughput: 0: 1793.8, 1: 1768.8. Samples: 38660648. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:47:53,441][51532] Avg episode reward: [(0, '67.210'), (1, '59.490')] -[2023-10-15 17:47:53,453][52833] Updated weights for policy 0, policy_version 75370 (0.0007) -[2023-10-15 17:47:53,832][52833] Updated weights for policy 0, policy_version 75380 (0.0008) -[2023-10-15 17:47:54,202][52833] Updated weights for policy 0, policy_version 75390 (0.0010) -[2023-10-15 17:47:54,461][52866] Updated weights for policy 1, policy_version 75620 (0.0009) -[2023-10-15 17:47:54,845][52866] Updated weights for policy 1, policy_version 75630 (0.0007) -[2023-10-15 17:47:55,215][52866] Updated weights for policy 1, policy_version 75640 (0.0011) -[2023-10-15 17:47:57,998][52833] Updated weights for policy 0, policy_version 75400 (0.0008) -[2023-10-15 17:47:58,371][52833] Updated weights for policy 0, policy_version 75410 (0.0008) -[2023-10-15 17:47:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 154664960. Throughput: 0: 1810.9, 1: 1770.1. Samples: 38682834. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:47:58,442][51532] Avg episode reward: [(0, '68.400'), (1, '60.820')] -[2023-10-15 17:47:58,743][52833] Updated weights for policy 0, policy_version 75420 (0.0009) -[2023-10-15 17:47:58,904][52866] Updated weights for policy 1, policy_version 75650 (0.0009) -[2023-10-15 17:47:59,278][52866] Updated weights for policy 1, policy_version 75660 (0.0008) -[2023-10-15 17:47:59,648][52866] Updated weights for policy 1, policy_version 75670 (0.0007) -[2023-10-15 17:48:00,014][52866] Updated weights for policy 1, policy_version 75680 (0.0009) -[2023-10-15 17:48:02,333][52833] Updated weights for policy 0, policy_version 75430 (0.0008) -[2023-10-15 17:48:02,697][52833] Updated weights for policy 0, policy_version 75440 (0.0008) -[2023-10-15 17:48:03,055][52833] Updated weights for policy 0, policy_version 75450 (0.0008) -[2023-10-15 17:48:03,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 154763264. Throughput: 0: 1802.8, 1: 1763.3. Samples: 38692952. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:48:03,442][51532] Avg episode reward: [(0, '64.570'), (1, '61.740')] -[2023-10-15 17:48:03,735][52866] Updated weights for policy 1, policy_version 75690 (0.0007) -[2023-10-15 17:48:04,089][52866] Updated weights for policy 1, policy_version 75700 (0.0011) -[2023-10-15 17:48:04,462][52866] Updated weights for policy 1, policy_version 75710 (0.0009) -[2023-10-15 17:48:06,611][52833] Updated weights for policy 0, policy_version 75460 (0.0010) -[2023-10-15 17:48:06,989][52833] Updated weights for policy 0, policy_version 75470 (0.0010) -[2023-10-15 17:48:07,357][52833] Updated weights for policy 0, policy_version 75480 (0.0009) -[2023-10-15 17:48:08,166][52866] Updated weights for policy 1, policy_version 75720 (0.0008) -[2023-10-15 17:48:08,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 154828800. Throughput: 0: 1811.6, 1: 1772.8. Samples: 38715410. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:48:08,441][51532] Avg episode reward: [(0, '66.230'), (1, '63.890')] -[2023-10-15 17:48:08,536][52866] Updated weights for policy 1, policy_version 75730 (0.0007) -[2023-10-15 17:48:08,916][52866] Updated weights for policy 1, policy_version 75740 (0.0008) -[2023-10-15 17:48:11,083][52833] Updated weights for policy 0, policy_version 75490 (0.0008) -[2023-10-15 17:48:11,450][52833] Updated weights for policy 0, policy_version 75500 (0.0008) -[2023-10-15 17:48:11,816][52833] Updated weights for policy 0, policy_version 75510 (0.0009) -[2023-10-15 17:48:12,183][52833] Updated weights for policy 0, policy_version 75520 (0.0010) -[2023-10-15 17:48:12,650][52866] Updated weights for policy 1, policy_version 75750 (0.0008) -[2023-10-15 17:48:13,014][52866] Updated weights for policy 1, policy_version 75760 (0.0008) -[2023-10-15 17:48:13,382][52866] Updated weights for policy 1, policy_version 75770 (0.0010) -[2023-10-15 17:48:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 154894336. Throughput: 0: 1809.9, 1: 1799.5. Samples: 38736476. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:48:13,442][51532] Avg episode reward: [(0, '67.760'), (1, '61.050')] -[2023-10-15 17:48:16,101][52833] Updated weights for policy 0, policy_version 75530 (0.0009) -[2023-10-15 17:48:16,471][52833] Updated weights for policy 0, policy_version 75540 (0.0008) -[2023-10-15 17:48:16,843][52833] Updated weights for policy 0, policy_version 75550 (0.0007) -[2023-10-15 17:48:17,070][52866] Updated weights for policy 1, policy_version 75780 (0.0009) -[2023-10-15 17:48:17,439][52866] Updated weights for policy 1, policy_version 75790 (0.0008) -[2023-10-15 17:48:17,808][52866] Updated weights for policy 1, policy_version 75800 (0.0007) -[2023-10-15 17:48:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154992640. Throughput: 0: 1815.5, 1: 1783.5. Samples: 38748412. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:48:18,441][51532] Avg episode reward: [(0, '68.790'), (1, '58.270')] -[2023-10-15 17:48:20,419][52833] Updated weights for policy 0, policy_version 75560 (0.0007) -[2023-10-15 17:48:20,783][52833] Updated weights for policy 0, policy_version 75570 (0.0007) -[2023-10-15 17:48:21,142][52833] Updated weights for policy 0, policy_version 75580 (0.0007) -[2023-10-15 17:48:21,563][52866] Updated weights for policy 1, policy_version 75810 (0.0008) -[2023-10-15 17:48:21,930][52866] Updated weights for policy 1, policy_version 75820 (0.0007) -[2023-10-15 17:48:22,302][52866] Updated weights for policy 1, policy_version 75830 (0.0010) -[2023-10-15 17:48:22,663][52866] Updated weights for policy 1, policy_version 75840 (0.0011) -[2023-10-15 17:48:23,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 155058176. Throughput: 0: 1814.1, 1: 1799.0. Samples: 38769096. Policy #0 lag: (min: 24.0, avg: 45.9, max: 48.0) -[2023-10-15 17:48:23,441][51532] Avg episode reward: [(0, '69.500'), (1, '55.100')] -[2023-10-15 17:48:24,928][52833] Updated weights for policy 0, policy_version 75590 (0.0010) -[2023-10-15 17:48:25,299][52833] Updated weights for policy 0, policy_version 75600 (0.0009) -[2023-10-15 17:48:25,680][52833] Updated weights for policy 0, policy_version 75610 (0.0011) -[2023-10-15 17:48:26,193][52866] Updated weights for policy 1, policy_version 75850 (0.0008) -[2023-10-15 17:48:26,562][52866] Updated weights for policy 1, policy_version 75860 (0.0010) -[2023-10-15 17:48:26,933][52866] Updated weights for policy 1, policy_version 75870 (0.0011) -[2023-10-15 17:48:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 155123712. Throughput: 0: 1812.5, 1: 1790.1. Samples: 38790970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:48:28,442][51532] Avg episode reward: [(0, '68.620'), (1, '56.380')] -[2023-10-15 17:48:29,618][52833] Updated weights for policy 0, policy_version 75620 (0.0008) -[2023-10-15 17:48:30,009][52833] Updated weights for policy 0, policy_version 75630 (0.0007) -[2023-10-15 17:48:30,372][52833] Updated weights for policy 0, policy_version 75640 (0.0009) -[2023-10-15 17:48:30,658][52866] Updated weights for policy 1, policy_version 75880 (0.0010) -[2023-10-15 17:48:31,026][52866] Updated weights for policy 1, policy_version 75890 (0.0010) -[2023-10-15 17:48:31,393][52866] Updated weights for policy 1, policy_version 75900 (0.0010) -[2023-10-15 17:48:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 155189248. Throughput: 0: 1807.3, 1: 1803.9. Samples: 38801262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:48:33,441][51532] Avg episode reward: [(0, '68.480'), (1, '56.360')] -[2023-10-15 17:48:33,951][52833] Updated weights for policy 0, policy_version 75650 (0.0008) -[2023-10-15 17:48:34,331][52833] Updated weights for policy 0, policy_version 75660 (0.0009) -[2023-10-15 17:48:34,698][52833] Updated weights for policy 0, policy_version 75670 (0.0008) -[2023-10-15 17:48:35,062][52833] Updated weights for policy 0, policy_version 75680 (0.0009) -[2023-10-15 17:48:35,088][52866] Updated weights for policy 1, policy_version 75910 (0.0009) -[2023-10-15 17:48:35,449][52866] Updated weights for policy 1, policy_version 75920 (0.0008) -[2023-10-15 17:48:35,807][52866] Updated weights for policy 1, policy_version 75930 (0.0008) -[2023-10-15 17:48:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 155254784. Throughput: 0: 1811.9, 1: 1801.4. Samples: 38823246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:48:38,442][51532] Avg episode reward: [(0, '71.560'), (1, '56.730')] -[2023-10-15 17:48:38,749][52833] Updated weights for policy 0, policy_version 75690 (0.0008) -[2023-10-15 17:48:39,116][52833] Updated weights for policy 0, policy_version 75700 (0.0007) -[2023-10-15 17:48:39,485][52833] Updated weights for policy 0, policy_version 75710 (0.0007) -[2023-10-15 17:48:39,603][52866] Updated weights for policy 1, policy_version 75940 (0.0010) -[2023-10-15 17:48:39,984][52866] Updated weights for policy 1, policy_version 75950 (0.0011) -[2023-10-15 17:48:40,361][52866] Updated weights for policy 1, policy_version 75960 (0.0009) -[2023-10-15 17:48:43,394][52833] Updated weights for policy 0, policy_version 75720 (0.0008) -[2023-10-15 17:48:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 155320320. Throughput: 0: 1814.2, 1: 1803.2. Samples: 38845618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:48:43,442][51532] Avg episode reward: [(0, '69.170'), (1, '59.580')] -[2023-10-15 17:48:43,763][52833] Updated weights for policy 0, policy_version 75730 (0.0008) -[2023-10-15 17:48:44,057][52866] Updated weights for policy 1, policy_version 75970 (0.0008) -[2023-10-15 17:48:44,135][52833] Updated weights for policy 0, policy_version 75740 (0.0010) -[2023-10-15 17:48:44,426][52866] Updated weights for policy 1, policy_version 75980 (0.0008) -[2023-10-15 17:48:44,794][52866] Updated weights for policy 1, policy_version 75990 (0.0008) -[2023-10-15 17:48:45,157][52866] Updated weights for policy 1, policy_version 76000 (0.0007) -[2023-10-15 17:48:47,684][52833] Updated weights for policy 0, policy_version 75750 (0.0010) -[2023-10-15 17:48:48,059][52833] Updated weights for policy 0, policy_version 75760 (0.0009) -[2023-10-15 17:48:48,428][52833] Updated weights for policy 0, policy_version 75770 (0.0008) -[2023-10-15 17:48:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 155385856. Throughput: 0: 1805.3, 1: 1806.2. Samples: 38855470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:48:48,441][51532] Avg episode reward: [(0, '68.450'), (1, '59.360')] -[2023-10-15 17:48:48,934][52866] Updated weights for policy 1, policy_version 76010 (0.0008) -[2023-10-15 17:48:49,299][52866] Updated weights for policy 1, policy_version 76020 (0.0007) -[2023-10-15 17:48:49,663][52866] Updated weights for policy 1, policy_version 76030 (0.0009) -[2023-10-15 17:48:51,964][52833] Updated weights for policy 0, policy_version 75780 (0.0008) -[2023-10-15 17:48:52,330][52833] Updated weights for policy 0, policy_version 75790 (0.0007) -[2023-10-15 17:48:52,699][52833] Updated weights for policy 0, policy_version 75800 (0.0008) -[2023-10-15 17:48:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 155484160. Throughput: 0: 1811.4, 1: 1801.0. Samples: 38877966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:48:53,442][51532] Avg episode reward: [(0, '70.040'), (1, '58.650')] -[2023-10-15 17:48:53,516][52866] Updated weights for policy 1, policy_version 76040 (0.0008) -[2023-10-15 17:48:53,884][52866] Updated weights for policy 1, policy_version 76050 (0.0008) -[2023-10-15 17:48:54,265][52866] Updated weights for policy 1, policy_version 76060 (0.0008) -[2023-10-15 17:48:56,464][52833] Updated weights for policy 0, policy_version 75810 (0.0009) -[2023-10-15 17:48:56,819][52833] Updated weights for policy 0, policy_version 75820 (0.0009) -[2023-10-15 17:48:57,187][52833] Updated weights for policy 0, policy_version 75830 (0.0007) -[2023-10-15 17:48:57,560][52833] Updated weights for policy 0, policy_version 75840 (0.0009) -[2023-10-15 17:48:58,148][52866] Updated weights for policy 1, policy_version 76070 (0.0010) -[2023-10-15 17:48:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 155549696. Throughput: 0: 1802.1, 1: 1813.5. Samples: 38899178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:48:58,441][51532] Avg episode reward: [(0, '68.720'), (1, '56.200')] -[2023-10-15 17:48:58,523][52866] Updated weights for policy 1, policy_version 76080 (0.0010) -[2023-10-15 17:48:58,894][52866] Updated weights for policy 1, policy_version 76090 (0.0008) -[2023-10-15 17:49:01,375][52833] Updated weights for policy 0, policy_version 75850 (0.0009) -[2023-10-15 17:49:01,739][52833] Updated weights for policy 0, policy_version 75860 (0.0009) -[2023-10-15 17:49:02,112][52833] Updated weights for policy 0, policy_version 75870 (0.0008) -[2023-10-15 17:49:02,575][52866] Updated weights for policy 1, policy_version 76100 (0.0009) -[2023-10-15 17:49:02,933][52866] Updated weights for policy 1, policy_version 76110 (0.0007) -[2023-10-15 17:49:03,306][52866] Updated weights for policy 1, policy_version 76120 (0.0008) -[2023-10-15 17:49:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 155615232. Throughput: 0: 1803.9, 1: 1799.6. Samples: 38910566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:49:03,441][51532] Avg episode reward: [(0, '72.350'), (1, '56.630')] -[2023-10-15 17:49:06,038][52833] Updated weights for policy 0, policy_version 75880 (0.0008) -[2023-10-15 17:49:06,398][52833] Updated weights for policy 0, policy_version 75890 (0.0008) -[2023-10-15 17:49:06,767][52833] Updated weights for policy 0, policy_version 75900 (0.0009) -[2023-10-15 17:49:07,001][52866] Updated weights for policy 1, policy_version 76130 (0.0010) -[2023-10-15 17:49:07,366][52866] Updated weights for policy 1, policy_version 76140 (0.0007) -[2023-10-15 17:49:07,731][52866] Updated weights for policy 1, policy_version 76150 (0.0009) -[2023-10-15 17:49:08,093][52866] Updated weights for policy 1, policy_version 76160 (0.0008) -[2023-10-15 17:49:08,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 155713536. Throughput: 0: 1795.5, 1: 1818.9. Samples: 38931744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:49:08,441][51532] Avg episode reward: [(0, '70.350'), (1, '59.150')] -[2023-10-15 17:49:10,534][52833] Updated weights for policy 0, policy_version 75910 (0.0009) -[2023-10-15 17:49:10,891][52833] Updated weights for policy 0, policy_version 75920 (0.0008) -[2023-10-15 17:49:11,270][52833] Updated weights for policy 0, policy_version 75930 (0.0008) -[2023-10-15 17:49:11,686][52866] Updated weights for policy 1, policy_version 76170 (0.0009) -[2023-10-15 17:49:12,049][52866] Updated weights for policy 1, policy_version 76180 (0.0011) -[2023-10-15 17:49:12,411][52866] Updated weights for policy 1, policy_version 76190 (0.0010) -[2023-10-15 17:49:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 155779072. Throughput: 0: 1800.2, 1: 1808.6. Samples: 38953366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:49:13,441][51532] Avg episode reward: [(0, '69.270'), (1, '59.810')] -[2023-10-15 17:49:14,883][52833] Updated weights for policy 0, policy_version 75940 (0.0008) -[2023-10-15 17:49:15,277][52833] Updated weights for policy 0, policy_version 75950 (0.0011) -[2023-10-15 17:49:15,648][52833] Updated weights for policy 0, policy_version 75960 (0.0009) -[2023-10-15 17:49:16,109][52866] Updated weights for policy 1, policy_version 76200 (0.0009) -[2023-10-15 17:49:16,482][52866] Updated weights for policy 1, policy_version 76210 (0.0009) -[2023-10-15 17:49:16,855][52866] Updated weights for policy 1, policy_version 76220 (0.0008) -[2023-10-15 17:49:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 155844608. Throughput: 0: 1807.6, 1: 1825.6. Samples: 38964756. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:18,442][51532] Avg episode reward: [(0, '71.470'), (1, '61.740')] -[2023-10-15 17:49:19,439][52833] Updated weights for policy 0, policy_version 75970 (0.0007) -[2023-10-15 17:49:19,809][52833] Updated weights for policy 0, policy_version 75980 (0.0008) -[2023-10-15 17:49:20,178][52833] Updated weights for policy 0, policy_version 75990 (0.0009) -[2023-10-15 17:49:20,547][52833] Updated weights for policy 0, policy_version 76000 (0.0010) -[2023-10-15 17:49:20,700][52866] Updated weights for policy 1, policy_version 76230 (0.0007) -[2023-10-15 17:49:21,068][52866] Updated weights for policy 1, policy_version 76240 (0.0008) -[2023-10-15 17:49:21,423][52866] Updated weights for policy 1, policy_version 76250 (0.0008) -[2023-10-15 17:49:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 155910144. Throughput: 0: 1798.1, 1: 1806.8. Samples: 38985462. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:23,441][51532] Avg episode reward: [(0, '72.800'), (1, '60.210')] -[2023-10-15 17:49:24,280][52833] Updated weights for policy 0, policy_version 76010 (0.0009) -[2023-10-15 17:49:24,639][52833] Updated weights for policy 0, policy_version 76020 (0.0007) -[2023-10-15 17:49:25,022][52833] Updated weights for policy 0, policy_version 76030 (0.0011) -[2023-10-15 17:49:25,250][52866] Updated weights for policy 1, policy_version 76260 (0.0010) -[2023-10-15 17:49:25,640][52866] Updated weights for policy 1, policy_version 76270 (0.0008) -[2023-10-15 17:49:26,008][52866] Updated weights for policy 1, policy_version 76280 (0.0009) -[2023-10-15 17:49:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 155975680. Throughput: 0: 1800.3, 1: 1807.1. Samples: 39007952. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:28,442][51532] Avg episode reward: [(0, '72.330'), (1, '61.060')] -[2023-10-15 17:49:28,769][52833] Updated weights for policy 0, policy_version 76040 (0.0011) -[2023-10-15 17:49:29,153][52833] Updated weights for policy 0, policy_version 76050 (0.0012) -[2023-10-15 17:49:29,525][52833] Updated weights for policy 0, policy_version 76060 (0.0008) -[2023-10-15 17:49:29,805][52866] Updated weights for policy 1, policy_version 76290 (0.0009) -[2023-10-15 17:49:30,164][52866] Updated weights for policy 1, policy_version 76300 (0.0008) -[2023-10-15 17:49:30,529][52866] Updated weights for policy 1, policy_version 76310 (0.0007) -[2023-10-15 17:49:30,886][52866] Updated weights for policy 1, policy_version 76320 (0.0010) -[2023-10-15 17:49:33,162][52833] Updated weights for policy 0, policy_version 76070 (0.0009) -[2023-10-15 17:49:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 156041216. Throughput: 0: 1802.8, 1: 1804.5. Samples: 39017800. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:33,441][51532] Avg episode reward: [(0, '73.140'), (1, '63.060')] -[2023-10-15 17:49:33,530][52833] Updated weights for policy 0, policy_version 76080 (0.0007) -[2023-10-15 17:49:33,887][52833] Updated weights for policy 0, policy_version 76090 (0.0010) -[2023-10-15 17:49:34,622][52866] Updated weights for policy 1, policy_version 76330 (0.0007) -[2023-10-15 17:49:34,990][52866] Updated weights for policy 1, policy_version 76340 (0.0010) -[2023-10-15 17:49:35,351][52866] Updated weights for policy 1, policy_version 76350 (0.0009) -[2023-10-15 17:49:37,604][52833] Updated weights for policy 0, policy_version 76100 (0.0009) -[2023-10-15 17:49:37,968][52833] Updated weights for policy 0, policy_version 76110 (0.0010) -[2023-10-15 17:49:38,336][52833] Updated weights for policy 0, policy_version 76120 (0.0007) -[2023-10-15 17:49:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 156106752. Throughput: 0: 1800.8, 1: 1805.7. Samples: 39040260. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:38,441][51532] Avg episode reward: [(0, '75.580'), (1, '62.670')] -[2023-10-15 17:49:39,062][52866] Updated weights for policy 1, policy_version 76360 (0.0007) -[2023-10-15 17:49:39,426][52866] Updated weights for policy 1, policy_version 76370 (0.0009) -[2023-10-15 17:49:39,788][52866] Updated weights for policy 1, policy_version 76380 (0.0009) -[2023-10-15 17:49:41,959][52833] Updated weights for policy 0, policy_version 76130 (0.0007) -[2023-10-15 17:49:42,339][52833] Updated weights for policy 0, policy_version 76140 (0.0009) -[2023-10-15 17:49:42,713][52833] Updated weights for policy 0, policy_version 76150 (0.0007) -[2023-10-15 17:49:43,090][52833] Updated weights for policy 0, policy_version 76160 (0.0007) -[2023-10-15 17:49:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 156205056. Throughput: 0: 1809.6, 1: 1814.7. Samples: 39062272. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:43,442][51532] Avg episode reward: [(0, '75.390'), (1, '66.410')] -[2023-10-15 17:49:43,443][52866] Updated weights for policy 1, policy_version 76390 (0.0008) -[2023-10-15 17:49:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000076160_77987840.pth... -[2023-10-15 17:49:43,490][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000074464_76251136.pth -[2023-10-15 17:49:43,813][52866] Updated weights for policy 1, policy_version 76400 (0.0007) -[2023-10-15 17:49:44,178][52866] Updated weights for policy 1, policy_version 76410 (0.0008) -[2023-10-15 17:49:44,402][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000076416_78249984.pth... -[2023-10-15 17:49:44,442][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000074720_76513280.pth -[2023-10-15 17:49:46,780][52833] Updated weights for policy 0, policy_version 76170 (0.0008) -[2023-10-15 17:49:47,154][52833] Updated weights for policy 0, policy_version 76180 (0.0011) -[2023-10-15 17:49:47,530][52833] Updated weights for policy 0, policy_version 76190 (0.0009) -[2023-10-15 17:49:48,030][52866] Updated weights for policy 1, policy_version 76420 (0.0008) -[2023-10-15 17:49:48,401][52866] Updated weights for policy 1, policy_version 76430 (0.0008) -[2023-10-15 17:49:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 156270592. Throughput: 0: 1801.2, 1: 1810.1. Samples: 39073072. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:48,442][51532] Avg episode reward: [(0, '74.390'), (1, '65.040')] -[2023-10-15 17:49:48,761][52866] Updated weights for policy 1, policy_version 76440 (0.0008) -[2023-10-15 17:49:51,343][52833] Updated weights for policy 0, policy_version 76200 (0.0008) -[2023-10-15 17:49:51,719][52833] Updated weights for policy 0, policy_version 76210 (0.0007) -[2023-10-15 17:49:52,076][52833] Updated weights for policy 0, policy_version 76220 (0.0007) -[2023-10-15 17:49:52,423][52866] Updated weights for policy 1, policy_version 76450 (0.0007) -[2023-10-15 17:49:52,783][52866] Updated weights for policy 1, policy_version 76460 (0.0009) -[2023-10-15 17:49:53,155][52866] Updated weights for policy 1, policy_version 76470 (0.0010) -[2023-10-15 17:49:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 156336128. Throughput: 0: 1808.9, 1: 1800.9. Samples: 39094186. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:53,442][51532] Avg episode reward: [(0, '75.540'), (1, '66.510')] -[2023-10-15 17:49:53,525][52866] Updated weights for policy 1, policy_version 76480 (0.0008) -[2023-10-15 17:49:55,855][52833] Updated weights for policy 0, policy_version 76230 (0.0009) -[2023-10-15 17:49:56,221][52833] Updated weights for policy 0, policy_version 76240 (0.0008) -[2023-10-15 17:49:56,600][52833] Updated weights for policy 0, policy_version 76250 (0.0009) -[2023-10-15 17:49:57,367][52866] Updated weights for policy 1, policy_version 76490 (0.0009) -[2023-10-15 17:49:57,730][52866] Updated weights for policy 1, policy_version 76500 (0.0010) -[2023-10-15 17:49:58,095][52866] Updated weights for policy 1, policy_version 76510 (0.0011) -[2023-10-15 17:49:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 156434432. Throughput: 0: 1787.3, 1: 1800.3. Samples: 39114808. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:49:58,442][51532] Avg episode reward: [(0, '74.480'), (1, '68.260')] -[2023-10-15 17:50:00,421][52833] Updated weights for policy 0, policy_version 76260 (0.0009) -[2023-10-15 17:50:00,808][52833] Updated weights for policy 0, policy_version 76270 (0.0007) -[2023-10-15 17:50:01,178][52833] Updated weights for policy 0, policy_version 76280 (0.0008) -[2023-10-15 17:50:01,816][52866] Updated weights for policy 1, policy_version 76520 (0.0008) -[2023-10-15 17:50:02,186][52866] Updated weights for policy 1, policy_version 76530 (0.0009) -[2023-10-15 17:50:02,553][52866] Updated weights for policy 1, policy_version 76540 (0.0009) -[2023-10-15 17:50:03,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 156499968. Throughput: 0: 1801.1, 1: 1793.3. Samples: 39126502. Policy #0 lag: (min: 35.0, avg: 54.4, max: 56.0) -[2023-10-15 17:50:03,442][51532] Avg episode reward: [(0, '76.000'), (1, '67.980')] -[2023-10-15 17:50:03,443][52410] Saving new best policy, reward=76.000! -[2023-10-15 17:50:04,871][52833] Updated weights for policy 0, policy_version 76290 (0.0008) -[2023-10-15 17:50:05,236][52833] Updated weights for policy 0, policy_version 76300 (0.0007) -[2023-10-15 17:50:05,597][52833] Updated weights for policy 0, policy_version 76310 (0.0009) -[2023-10-15 17:50:05,969][52833] Updated weights for policy 0, policy_version 76320 (0.0008) -[2023-10-15 17:50:06,285][52866] Updated weights for policy 1, policy_version 76550 (0.0009) -[2023-10-15 17:50:06,638][52866] Updated weights for policy 1, policy_version 76560 (0.0011) -[2023-10-15 17:50:07,001][52866] Updated weights for policy 1, policy_version 76570 (0.0009) -[2023-10-15 17:50:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 156565504. Throughput: 0: 1793.6, 1: 1799.1. Samples: 39147138. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:08,442][51532] Avg episode reward: [(0, '76.110'), (1, '68.190')] -[2023-10-15 17:50:08,443][52410] Saving new best policy, reward=76.110! -[2023-10-15 17:50:09,586][52833] Updated weights for policy 0, policy_version 76330 (0.0007) -[2023-10-15 17:50:09,956][52833] Updated weights for policy 0, policy_version 76340 (0.0007) -[2023-10-15 17:50:10,318][52833] Updated weights for policy 0, policy_version 76350 (0.0007) -[2023-10-15 17:50:10,830][52866] Updated weights for policy 1, policy_version 76580 (0.0009) -[2023-10-15 17:50:11,214][52866] Updated weights for policy 1, policy_version 76590 (0.0010) -[2023-10-15 17:50:11,584][52866] Updated weights for policy 1, policy_version 76600 (0.0009) -[2023-10-15 17:50:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156631040. Throughput: 0: 1803.7, 1: 1787.7. Samples: 39169564. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:13,441][51532] Avg episode reward: [(0, '74.940'), (1, '70.080')] -[2023-10-15 17:50:14,095][52833] Updated weights for policy 0, policy_version 76360 (0.0009) -[2023-10-15 17:50:14,468][52833] Updated weights for policy 0, policy_version 76370 (0.0009) -[2023-10-15 17:50:14,844][52833] Updated weights for policy 0, policy_version 76380 (0.0008) -[2023-10-15 17:50:15,219][52866] Updated weights for policy 1, policy_version 76610 (0.0008) -[2023-10-15 17:50:15,579][52866] Updated weights for policy 1, policy_version 76620 (0.0008) -[2023-10-15 17:50:15,940][52866] Updated weights for policy 1, policy_version 76630 (0.0007) -[2023-10-15 17:50:16,306][52866] Updated weights for policy 1, policy_version 76640 (0.0008) -[2023-10-15 17:50:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156696576. Throughput: 0: 1797.6, 1: 1804.2. Samples: 39179878. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:18,441][51532] Avg episode reward: [(0, '75.090'), (1, '70.330')] -[2023-10-15 17:50:18,697][52833] Updated weights for policy 0, policy_version 76390 (0.0009) -[2023-10-15 17:50:19,066][52833] Updated weights for policy 0, policy_version 76400 (0.0007) -[2023-10-15 17:50:19,432][52833] Updated weights for policy 0, policy_version 76410 (0.0008) -[2023-10-15 17:50:19,941][52866] Updated weights for policy 1, policy_version 76650 (0.0008) -[2023-10-15 17:50:20,308][52866] Updated weights for policy 1, policy_version 76660 (0.0010) -[2023-10-15 17:50:20,667][52866] Updated weights for policy 1, policy_version 76670 (0.0011) -[2023-10-15 17:50:23,233][52833] Updated weights for policy 0, policy_version 76420 (0.0008) -[2023-10-15 17:50:23,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 156762112. Throughput: 0: 1794.5, 1: 1797.8. Samples: 39201914. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:23,442][51532] Avg episode reward: [(0, '76.090'), (1, '71.930')] -[2023-10-15 17:50:23,599][52833] Updated weights for policy 0, policy_version 76430 (0.0007) -[2023-10-15 17:50:23,979][52833] Updated weights for policy 0, policy_version 76440 (0.0009) -[2023-10-15 17:50:24,378][52866] Updated weights for policy 1, policy_version 76680 (0.0010) -[2023-10-15 17:50:24,739][52866] Updated weights for policy 1, policy_version 76690 (0.0011) -[2023-10-15 17:50:25,105][52866] Updated weights for policy 1, policy_version 76700 (0.0010) -[2023-10-15 17:50:27,837][52833] Updated weights for policy 0, policy_version 76450 (0.0010) -[2023-10-15 17:50:28,197][52833] Updated weights for policy 0, policy_version 76460 (0.0007) -[2023-10-15 17:50:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 156827648. Throughput: 0: 1813.0, 1: 1786.6. Samples: 39224254. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:28,441][51532] Avg episode reward: [(0, '72.080'), (1, '73.410')] -[2023-10-15 17:50:28,573][52833] Updated weights for policy 0, policy_version 76470 (0.0007) -[2023-10-15 17:50:28,904][52866] Updated weights for policy 1, policy_version 76710 (0.0008) -[2023-10-15 17:50:28,936][52833] Updated weights for policy 0, policy_version 76480 (0.0007) -[2023-10-15 17:50:29,272][52866] Updated weights for policy 1, policy_version 76720 (0.0010) -[2023-10-15 17:50:29,647][52866] Updated weights for policy 1, policy_version 76730 (0.0012) -[2023-10-15 17:50:32,638][52833] Updated weights for policy 0, policy_version 76490 (0.0010) -[2023-10-15 17:50:33,017][52833] Updated weights for policy 0, policy_version 76500 (0.0011) -[2023-10-15 17:50:33,389][52833] Updated weights for policy 0, policy_version 76510 (0.0009) -[2023-10-15 17:50:33,413][52866] Updated weights for policy 1, policy_version 76740 (0.0009) -[2023-10-15 17:50:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 156893184. Throughput: 0: 1793.4, 1: 1786.2. Samples: 39234156. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:33,442][51532] Avg episode reward: [(0, '70.980'), (1, '73.650')] -[2023-10-15 17:50:33,781][52866] Updated weights for policy 1, policy_version 76750 (0.0008) -[2023-10-15 17:50:34,143][52866] Updated weights for policy 1, policy_version 76760 (0.0007) -[2023-10-15 17:50:36,782][52833] Updated weights for policy 0, policy_version 76520 (0.0010) -[2023-10-15 17:50:37,149][52833] Updated weights for policy 0, policy_version 76530 (0.0010) -[2023-10-15 17:50:37,516][52833] Updated weights for policy 0, policy_version 76540 (0.0009) -[2023-10-15 17:50:37,864][52866] Updated weights for policy 1, policy_version 76770 (0.0008) -[2023-10-15 17:50:38,233][52866] Updated weights for policy 1, policy_version 76780 (0.0008) -[2023-10-15 17:50:38,441][51532] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 156991488. Throughput: 0: 1812.5, 1: 1792.8. Samples: 39256424. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:38,442][51532] Avg episode reward: [(0, '69.170'), (1, '71.190')] -[2023-10-15 17:50:38,598][52866] Updated weights for policy 1, policy_version 76790 (0.0009) -[2023-10-15 17:50:38,966][52866] Updated weights for policy 1, policy_version 76800 (0.0009) -[2023-10-15 17:50:41,411][52833] Updated weights for policy 0, policy_version 76550 (0.0009) -[2023-10-15 17:50:41,780][52833] Updated weights for policy 0, policy_version 76560 (0.0010) -[2023-10-15 17:50:42,146][52833] Updated weights for policy 0, policy_version 76570 (0.0009) -[2023-10-15 17:50:42,619][52866] Updated weights for policy 1, policy_version 76810 (0.0008) -[2023-10-15 17:50:42,985][52866] Updated weights for policy 1, policy_version 76820 (0.0008) -[2023-10-15 17:50:43,348][52866] Updated weights for policy 1, policy_version 76830 (0.0012) -[2023-10-15 17:50:43,441][51532] Fps is (10 sec: 19660.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 157089792. Throughput: 0: 1804.8, 1: 1806.0. Samples: 39277298. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:43,442][51532] Avg episode reward: [(0, '68.740'), (1, '68.540')] -[2023-10-15 17:50:45,796][52833] Updated weights for policy 0, policy_version 76580 (0.0010) -[2023-10-15 17:50:46,179][52833] Updated weights for policy 0, policy_version 76590 (0.0010) -[2023-10-15 17:50:46,552][52833] Updated weights for policy 0, policy_version 76600 (0.0009) -[2023-10-15 17:50:47,184][52866] Updated weights for policy 1, policy_version 76840 (0.0008) -[2023-10-15 17:50:47,559][52866] Updated weights for policy 1, policy_version 76850 (0.0007) -[2023-10-15 17:50:47,920][52866] Updated weights for policy 1, policy_version 76860 (0.0008) -[2023-10-15 17:50:48,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 157155328. Throughput: 0: 1815.4, 1: 1799.3. Samples: 39289164. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:48,441][51532] Avg episode reward: [(0, '69.340'), (1, '65.340')] -[2023-10-15 17:50:50,305][52833] Updated weights for policy 0, policy_version 76610 (0.0009) -[2023-10-15 17:50:50,675][52833] Updated weights for policy 0, policy_version 76620 (0.0009) -[2023-10-15 17:50:51,036][52833] Updated weights for policy 0, policy_version 76630 (0.0009) -[2023-10-15 17:50:51,405][52833] Updated weights for policy 0, policy_version 76640 (0.0011) -[2023-10-15 17:50:51,703][52866] Updated weights for policy 1, policy_version 76870 (0.0008) -[2023-10-15 17:50:52,068][52866] Updated weights for policy 1, policy_version 76880 (0.0010) -[2023-10-15 17:50:52,433][52866] Updated weights for policy 1, policy_version 76890 (0.0011) -[2023-10-15 17:50:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 157220864. Throughput: 0: 1804.9, 1: 1812.6. Samples: 39309926. Policy #0 lag: (min: 31.0, avg: 32.6, max: 57.0) -[2023-10-15 17:50:53,442][51532] Avg episode reward: [(0, '69.190'), (1, '68.860')] -[2023-10-15 17:50:55,165][52833] Updated weights for policy 0, policy_version 76650 (0.0008) -[2023-10-15 17:50:55,533][52833] Updated weights for policy 0, policy_version 76660 (0.0008) -[2023-10-15 17:50:55,902][52833] Updated weights for policy 0, policy_version 76670 (0.0009) -[2023-10-15 17:50:56,400][52866] Updated weights for policy 1, policy_version 76900 (0.0011) -[2023-10-15 17:50:56,779][52866] Updated weights for policy 1, policy_version 76910 (0.0008) -[2023-10-15 17:50:57,137][52866] Updated weights for policy 1, policy_version 76920 (0.0008) -[2023-10-15 17:50:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157286400. Throughput: 0: 1795.5, 1: 1798.5. Samples: 39331296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:50:58,442][51532] Avg episode reward: [(0, '66.290'), (1, '67.660')] -[2023-10-15 17:50:59,585][52833] Updated weights for policy 0, policy_version 76680 (0.0008) -[2023-10-15 17:50:59,954][52833] Updated weights for policy 0, policy_version 76690 (0.0008) -[2023-10-15 17:51:00,319][52833] Updated weights for policy 0, policy_version 76700 (0.0008) -[2023-10-15 17:51:00,800][52866] Updated weights for policy 1, policy_version 76930 (0.0009) -[2023-10-15 17:51:01,165][52866] Updated weights for policy 1, policy_version 76940 (0.0009) -[2023-10-15 17:51:01,535][52866] Updated weights for policy 1, policy_version 76950 (0.0008) -[2023-10-15 17:51:01,898][52866] Updated weights for policy 1, policy_version 76960 (0.0010) -[2023-10-15 17:51:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157351936. Throughput: 0: 1801.3, 1: 1811.9. Samples: 39342472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:03,441][51532] Avg episode reward: [(0, '68.010'), (1, '65.220')] -[2023-10-15 17:51:03,919][52833] Updated weights for policy 0, policy_version 76710 (0.0009) -[2023-10-15 17:51:04,288][52833] Updated weights for policy 0, policy_version 76720 (0.0008) -[2023-10-15 17:51:04,657][52833] Updated weights for policy 0, policy_version 76730 (0.0008) -[2023-10-15 17:51:05,772][52866] Updated weights for policy 1, policy_version 76970 (0.0008) -[2023-10-15 17:51:06,133][52866] Updated weights for policy 1, policy_version 76980 (0.0007) -[2023-10-15 17:51:06,501][52866] Updated weights for policy 1, policy_version 76990 (0.0008) -[2023-10-15 17:51:08,392][52833] Updated weights for policy 0, policy_version 76740 (0.0009) -[2023-10-15 17:51:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157417472. Throughput: 0: 1808.9, 1: 1790.3. Samples: 39363876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:08,441][51532] Avg episode reward: [(0, '68.390'), (1, '66.390')] -[2023-10-15 17:51:08,768][52833] Updated weights for policy 0, policy_version 76750 (0.0011) -[2023-10-15 17:51:09,128][52833] Updated weights for policy 0, policy_version 76760 (0.0010) -[2023-10-15 17:51:10,049][52866] Updated weights for policy 1, policy_version 77000 (0.0008) -[2023-10-15 17:51:10,420][52866] Updated weights for policy 1, policy_version 77010 (0.0007) -[2023-10-15 17:51:10,781][52866] Updated weights for policy 1, policy_version 77020 (0.0007) -[2023-10-15 17:51:12,995][52833] Updated weights for policy 0, policy_version 76770 (0.0010) -[2023-10-15 17:51:13,371][52833] Updated weights for policy 0, policy_version 76780 (0.0009) -[2023-10-15 17:51:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157483008. Throughput: 0: 1809.1, 1: 1793.6. Samples: 39386378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:13,441][51532] Avg episode reward: [(0, '67.730'), (1, '66.420')] -[2023-10-15 17:51:13,740][52833] Updated weights for policy 0, policy_version 76790 (0.0008) -[2023-10-15 17:51:14,109][52833] Updated weights for policy 0, policy_version 76800 (0.0007) -[2023-10-15 17:51:14,512][52866] Updated weights for policy 1, policy_version 77030 (0.0007) -[2023-10-15 17:51:14,881][52866] Updated weights for policy 1, policy_version 77040 (0.0007) -[2023-10-15 17:51:15,248][52866] Updated weights for policy 1, policy_version 77050 (0.0007) -[2023-10-15 17:51:17,887][52833] Updated weights for policy 0, policy_version 76810 (0.0008) -[2023-10-15 17:51:18,251][52833] Updated weights for policy 0, policy_version 76820 (0.0009) -[2023-10-15 17:51:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 157548544. Throughput: 0: 1804.7, 1: 1795.1. Samples: 39396146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:18,442][51532] Avg episode reward: [(0, '65.590'), (1, '62.490')] -[2023-10-15 17:51:18,624][52833] Updated weights for policy 0, policy_version 76830 (0.0009) -[2023-10-15 17:51:19,170][52866] Updated weights for policy 1, policy_version 77060 (0.0007) -[2023-10-15 17:51:19,535][52866] Updated weights for policy 1, policy_version 77070 (0.0007) -[2023-10-15 17:51:19,907][52866] Updated weights for policy 1, policy_version 77080 (0.0008) -[2023-10-15 17:51:22,488][52833] Updated weights for policy 0, policy_version 76840 (0.0008) -[2023-10-15 17:51:22,860][52833] Updated weights for policy 0, policy_version 76850 (0.0007) -[2023-10-15 17:51:23,234][52833] Updated weights for policy 0, policy_version 76860 (0.0009) -[2023-10-15 17:51:23,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 157646848. Throughput: 0: 1808.3, 1: 1790.6. Samples: 39418372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:23,442][51532] Avg episode reward: [(0, '67.290'), (1, '59.620')] -[2023-10-15 17:51:23,581][52866] Updated weights for policy 1, policy_version 77090 (0.0008) -[2023-10-15 17:51:23,941][52866] Updated weights for policy 1, policy_version 77100 (0.0008) -[2023-10-15 17:51:24,290][52866] Updated weights for policy 1, policy_version 77110 (0.0008) -[2023-10-15 17:51:24,653][52866] Updated weights for policy 1, policy_version 77120 (0.0009) -[2023-10-15 17:51:27,039][52833] Updated weights for policy 0, policy_version 76870 (0.0008) -[2023-10-15 17:51:27,409][52833] Updated weights for policy 0, policy_version 76880 (0.0008) -[2023-10-15 17:51:27,775][52833] Updated weights for policy 0, policy_version 76890 (0.0009) -[2023-10-15 17:51:28,245][52866] Updated weights for policy 1, policy_version 77130 (0.0008) -[2023-10-15 17:51:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 157712384. Throughput: 0: 1803.7, 1: 1810.8. Samples: 39439948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:28,442][51532] Avg episode reward: [(0, '72.050'), (1, '59.330')] -[2023-10-15 17:51:28,613][52866] Updated weights for policy 1, policy_version 77140 (0.0010) -[2023-10-15 17:51:28,981][52866] Updated weights for policy 1, policy_version 77150 (0.0011) -[2023-10-15 17:51:31,594][52833] Updated weights for policy 0, policy_version 76900 (0.0009) -[2023-10-15 17:51:31,994][52833] Updated weights for policy 0, policy_version 76910 (0.0010) -[2023-10-15 17:51:32,373][52833] Updated weights for policy 0, policy_version 76920 (0.0009) -[2023-10-15 17:51:32,791][52866] Updated weights for policy 1, policy_version 77160 (0.0008) -[2023-10-15 17:51:33,157][52866] Updated weights for policy 1, policy_version 77170 (0.0007) -[2023-10-15 17:51:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 157777920. Throughput: 0: 1806.9, 1: 1792.1. Samples: 39451120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:33,441][51532] Avg episode reward: [(0, '71.800'), (1, '59.970')] -[2023-10-15 17:51:33,524][52866] Updated weights for policy 1, policy_version 77180 (0.0008) -[2023-10-15 17:51:36,091][52833] Updated weights for policy 0, policy_version 76930 (0.0008) -[2023-10-15 17:51:36,463][52833] Updated weights for policy 0, policy_version 76940 (0.0011) -[2023-10-15 17:51:36,835][52833] Updated weights for policy 0, policy_version 76950 (0.0011) -[2023-10-15 17:51:37,198][52833] Updated weights for policy 0, policy_version 76960 (0.0008) -[2023-10-15 17:51:37,207][52866] Updated weights for policy 1, policy_version 77190 (0.0009) -[2023-10-15 17:51:37,572][52866] Updated weights for policy 1, policy_version 77200 (0.0010) -[2023-10-15 17:51:37,935][52866] Updated weights for policy 1, policy_version 77210 (0.0010) -[2023-10-15 17:51:38,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 157876224. Throughput: 0: 1803.0, 1: 1806.0. Samples: 39472330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:38,442][51532] Avg episode reward: [(0, '71.260'), (1, '62.500')] -[2023-10-15 17:51:40,989][52833] Updated weights for policy 0, policy_version 76970 (0.0010) -[2023-10-15 17:51:41,360][52833] Updated weights for policy 0, policy_version 76980 (0.0008) -[2023-10-15 17:51:41,722][52833] Updated weights for policy 0, policy_version 76990 (0.0007) -[2023-10-15 17:51:41,834][52866] Updated weights for policy 1, policy_version 77220 (0.0009) -[2023-10-15 17:51:42,231][52866] Updated weights for policy 1, policy_version 77230 (0.0010) -[2023-10-15 17:51:42,605][52866] Updated weights for policy 1, policy_version 77240 (0.0009) -[2023-10-15 17:51:43,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157941760. Throughput: 0: 1792.7, 1: 1792.3. Samples: 39492622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:51:43,441][51532] Avg episode reward: [(0, '72.130'), (1, '65.120')] -[2023-10-15 17:51:43,449][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000077248_79101952.pth... -[2023-10-15 17:51:43,449][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000076992_78839808.pth... -[2023-10-15 17:51:43,480][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000075296_77103104.pth -[2023-10-15 17:51:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000075552_77365248.pth -[2023-10-15 17:51:45,417][52833] Updated weights for policy 0, policy_version 77000 (0.0008) -[2023-10-15 17:51:45,794][52833] Updated weights for policy 0, policy_version 77010 (0.0009) -[2023-10-15 17:51:46,170][52833] Updated weights for policy 0, policy_version 77020 (0.0009) -[2023-10-15 17:51:46,281][52866] Updated weights for policy 1, policy_version 77250 (0.0011) -[2023-10-15 17:51:46,640][52866] Updated weights for policy 1, policy_version 77260 (0.0011) -[2023-10-15 17:51:47,005][52866] Updated weights for policy 1, policy_version 77270 (0.0008) -[2023-10-15 17:51:47,376][52866] Updated weights for policy 1, policy_version 77280 (0.0008) -[2023-10-15 17:51:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 158007296. Throughput: 0: 1802.4, 1: 1798.9. Samples: 39504530. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:51:48,441][51532] Avg episode reward: [(0, '69.510'), (1, '64.520')] -[2023-10-15 17:51:49,857][52833] Updated weights for policy 0, policy_version 77030 (0.0010) -[2023-10-15 17:51:50,228][52833] Updated weights for policy 0, policy_version 77040 (0.0009) -[2023-10-15 17:51:50,594][52833] Updated weights for policy 0, policy_version 77050 (0.0008) -[2023-10-15 17:51:51,264][52866] Updated weights for policy 1, policy_version 77290 (0.0009) -[2023-10-15 17:51:51,629][52866] Updated weights for policy 1, policy_version 77300 (0.0008) -[2023-10-15 17:51:52,007][52866] Updated weights for policy 1, policy_version 77310 (0.0010) -[2023-10-15 17:51:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 158072832. Throughput: 0: 1784.0, 1: 1797.6. Samples: 39525044. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:51:53,441][51532] Avg episode reward: [(0, '70.750'), (1, '63.610')] -[2023-10-15 17:51:54,280][52833] Updated weights for policy 0, policy_version 77060 (0.0008) -[2023-10-15 17:51:54,652][52833] Updated weights for policy 0, policy_version 77070 (0.0009) -[2023-10-15 17:51:55,015][52833] Updated weights for policy 0, policy_version 77080 (0.0008) -[2023-10-15 17:51:55,712][52866] Updated weights for policy 1, policy_version 77320 (0.0009) -[2023-10-15 17:51:56,075][52866] Updated weights for policy 1, policy_version 77330 (0.0011) -[2023-10-15 17:51:56,447][52866] Updated weights for policy 1, policy_version 77340 (0.0008) -[2023-10-15 17:51:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 158138368. Throughput: 0: 1786.5, 1: 1792.3. Samples: 39547424. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:51:58,442][51532] Avg episode reward: [(0, '72.040'), (1, '64.020')] -[2023-10-15 17:51:58,735][52833] Updated weights for policy 0, policy_version 77090 (0.0008) -[2023-10-15 17:51:59,104][52833] Updated weights for policy 0, policy_version 77100 (0.0008) -[2023-10-15 17:51:59,481][52833] Updated weights for policy 0, policy_version 77110 (0.0007) -[2023-10-15 17:51:59,852][52833] Updated weights for policy 0, policy_version 77120 (0.0007) -[2023-10-15 17:52:00,242][52866] Updated weights for policy 1, policy_version 77350 (0.0009) -[2023-10-15 17:52:00,609][52866] Updated weights for policy 1, policy_version 77360 (0.0008) -[2023-10-15 17:52:00,970][52866] Updated weights for policy 1, policy_version 77370 (0.0007) -[2023-10-15 17:52:03,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 158203904. Throughput: 0: 1787.6, 1: 1798.0. Samples: 39557500. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:52:03,442][51532] Avg episode reward: [(0, '67.340'), (1, '65.800')] -[2023-10-15 17:52:03,571][52833] Updated weights for policy 0, policy_version 77130 (0.0008) -[2023-10-15 17:52:03,935][52833] Updated weights for policy 0, policy_version 77140 (0.0009) -[2023-10-15 17:52:04,305][52833] Updated weights for policy 0, policy_version 77150 (0.0009) -[2023-10-15 17:52:04,692][52866] Updated weights for policy 1, policy_version 77380 (0.0009) -[2023-10-15 17:52:05,053][52866] Updated weights for policy 1, policy_version 77390 (0.0011) -[2023-10-15 17:52:05,430][52866] Updated weights for policy 1, policy_version 77400 (0.0009) -[2023-10-15 17:52:08,094][52833] Updated weights for policy 0, policy_version 77160 (0.0007) -[2023-10-15 17:52:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 158269440. Throughput: 0: 1790.3, 1: 1795.9. Samples: 39579748. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:52:08,441][51532] Avg episode reward: [(0, '68.110'), (1, '66.230')] -[2023-10-15 17:52:08,460][52833] Updated weights for policy 0, policy_version 77170 (0.0007) -[2023-10-15 17:52:08,826][52833] Updated weights for policy 0, policy_version 77180 (0.0009) -[2023-10-15 17:52:09,134][52866] Updated weights for policy 1, policy_version 77410 (0.0008) -[2023-10-15 17:52:09,487][52866] Updated weights for policy 1, policy_version 77420 (0.0008) -[2023-10-15 17:52:09,851][52866] Updated weights for policy 1, policy_version 77430 (0.0007) -[2023-10-15 17:52:10,216][52866] Updated weights for policy 1, policy_version 77440 (0.0007) -[2023-10-15 17:52:12,479][52833] Updated weights for policy 0, policy_version 77190 (0.0008) -[2023-10-15 17:52:12,839][52833] Updated weights for policy 0, policy_version 77200 (0.0007) -[2023-10-15 17:52:13,217][52833] Updated weights for policy 0, policy_version 77210 (0.0008) -[2023-10-15 17:52:13,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 158367744. Throughput: 0: 1800.9, 1: 1795.8. Samples: 39601798. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:52:13,441][51532] Avg episode reward: [(0, '66.080'), (1, '63.780')] -[2023-10-15 17:52:13,890][52866] Updated weights for policy 1, policy_version 77450 (0.0009) -[2023-10-15 17:52:14,247][52866] Updated weights for policy 1, policy_version 77460 (0.0009) -[2023-10-15 17:52:14,619][52866] Updated weights for policy 1, policy_version 77470 (0.0008) -[2023-10-15 17:52:16,980][52833] Updated weights for policy 0, policy_version 77220 (0.0008) -[2023-10-15 17:52:17,366][52833] Updated weights for policy 0, policy_version 77230 (0.0008) -[2023-10-15 17:52:17,729][52833] Updated weights for policy 0, policy_version 77240 (0.0010) -[2023-10-15 17:52:18,256][52866] Updated weights for policy 1, policy_version 77480 (0.0010) -[2023-10-15 17:52:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 158433280. Throughput: 0: 1790.7, 1: 1796.0. Samples: 39612522. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:52:18,441][51532] Avg episode reward: [(0, '63.520'), (1, '63.910')] -[2023-10-15 17:52:18,629][52866] Updated weights for policy 1, policy_version 77490 (0.0007) -[2023-10-15 17:52:18,998][52866] Updated weights for policy 1, policy_version 77500 (0.0011) -[2023-10-15 17:52:21,401][52833] Updated weights for policy 0, policy_version 77250 (0.0009) -[2023-10-15 17:52:21,768][52833] Updated weights for policy 0, policy_version 77260 (0.0008) -[2023-10-15 17:52:22,143][52833] Updated weights for policy 0, policy_version 77270 (0.0007) -[2023-10-15 17:52:22,511][52833] Updated weights for policy 0, policy_version 77280 (0.0008) -[2023-10-15 17:52:22,707][52866] Updated weights for policy 1, policy_version 77510 (0.0009) -[2023-10-15 17:52:23,068][52866] Updated weights for policy 1, policy_version 77520 (0.0007) -[2023-10-15 17:52:23,441][52866] Updated weights for policy 1, policy_version 77530 (0.0008) -[2023-10-15 17:52:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 158498816. Throughput: 0: 1803.8, 1: 1799.6. Samples: 39634486. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:52:23,442][51532] Avg episode reward: [(0, '61.470'), (1, '68.860')] -[2023-10-15 17:52:26,222][52833] Updated weights for policy 0, policy_version 77290 (0.0011) -[2023-10-15 17:52:26,596][52833] Updated weights for policy 0, policy_version 77300 (0.0009) -[2023-10-15 17:52:26,950][52833] Updated weights for policy 0, policy_version 77310 (0.0010) -[2023-10-15 17:52:27,125][52866] Updated weights for policy 1, policy_version 77540 (0.0009) -[2023-10-15 17:52:27,511][52866] Updated weights for policy 1, policy_version 77550 (0.0009) -[2023-10-15 17:52:27,875][52866] Updated weights for policy 1, policy_version 77560 (0.0009) -[2023-10-15 17:52:28,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 158597120. Throughput: 0: 1798.8, 1: 1812.6. Samples: 39655136. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:52:28,442][51532] Avg episode reward: [(0, '60.320'), (1, '73.860')] -[2023-10-15 17:52:30,611][52833] Updated weights for policy 0, policy_version 77320 (0.0009) -[2023-10-15 17:52:30,971][52833] Updated weights for policy 0, policy_version 77330 (0.0010) -[2023-10-15 17:52:31,344][52833] Updated weights for policy 0, policy_version 77340 (0.0010) -[2023-10-15 17:52:31,675][52866] Updated weights for policy 1, policy_version 77570 (0.0010) -[2023-10-15 17:52:32,047][52866] Updated weights for policy 1, policy_version 77580 (0.0008) -[2023-10-15 17:52:32,412][52866] Updated weights for policy 1, policy_version 77590 (0.0009) -[2023-10-15 17:52:32,780][52866] Updated weights for policy 1, policy_version 77600 (0.0010) -[2023-10-15 17:52:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 158662656. Throughput: 0: 1809.1, 1: 1799.8. Samples: 39666934. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 17:52:33,442][51532] Avg episode reward: [(0, '61.970'), (1, '72.640')] -[2023-10-15 17:52:35,028][52833] Updated weights for policy 0, policy_version 77350 (0.0009) -[2023-10-15 17:52:35,394][52833] Updated weights for policy 0, policy_version 77360 (0.0008) -[2023-10-15 17:52:35,771][52833] Updated weights for policy 0, policy_version 77370 (0.0009) -[2023-10-15 17:52:36,615][52866] Updated weights for policy 1, policy_version 77610 (0.0008) -[2023-10-15 17:52:36,977][52866] Updated weights for policy 1, policy_version 77620 (0.0008) -[2023-10-15 17:52:37,347][52866] Updated weights for policy 1, policy_version 77630 (0.0008) -[2023-10-15 17:52:38,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 158728192. Throughput: 0: 1805.1, 1: 1808.4. Samples: 39687654. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:52:38,442][51532] Avg episode reward: [(0, '61.550'), (1, '75.170')] -[2023-10-15 17:52:39,608][52833] Updated weights for policy 0, policy_version 77380 (0.0008) -[2023-10-15 17:52:39,973][52833] Updated weights for policy 0, policy_version 77390 (0.0009) -[2023-10-15 17:52:40,347][52833] Updated weights for policy 0, policy_version 77400 (0.0007) -[2023-10-15 17:52:41,093][52866] Updated weights for policy 1, policy_version 77640 (0.0008) -[2023-10-15 17:52:41,459][52866] Updated weights for policy 1, policy_version 77650 (0.0008) -[2023-10-15 17:52:41,832][52866] Updated weights for policy 1, policy_version 77660 (0.0009) -[2023-10-15 17:52:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 158793728. Throughput: 0: 1807.2, 1: 1794.0. Samples: 39709476. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:52:43,442][51532] Avg episode reward: [(0, '62.900'), (1, '74.830')] -[2023-10-15 17:52:44,061][52833] Updated weights for policy 0, policy_version 77410 (0.0008) -[2023-10-15 17:52:44,433][52833] Updated weights for policy 0, policy_version 77420 (0.0007) -[2023-10-15 17:52:44,809][52833] Updated weights for policy 0, policy_version 77430 (0.0007) -[2023-10-15 17:52:45,180][52833] Updated weights for policy 0, policy_version 77440 (0.0010) -[2023-10-15 17:52:45,641][52866] Updated weights for policy 1, policy_version 77670 (0.0007) -[2023-10-15 17:52:46,018][52866] Updated weights for policy 1, policy_version 77680 (0.0007) -[2023-10-15 17:52:46,380][52866] Updated weights for policy 1, policy_version 77690 (0.0009) -[2023-10-15 17:52:48,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 158859264. Throughput: 0: 1806.5, 1: 1806.4. Samples: 39720078. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:52:48,441][51532] Avg episode reward: [(0, '62.340'), (1, '73.460')] -[2023-10-15 17:52:48,792][52833] Updated weights for policy 0, policy_version 77450 (0.0011) -[2023-10-15 17:52:49,169][52833] Updated weights for policy 0, policy_version 77460 (0.0011) -[2023-10-15 17:52:49,544][52833] Updated weights for policy 0, policy_version 77470 (0.0010) -[2023-10-15 17:52:50,246][52866] Updated weights for policy 1, policy_version 77700 (0.0010) -[2023-10-15 17:52:50,606][52866] Updated weights for policy 1, policy_version 77710 (0.0010) -[2023-10-15 17:52:50,971][52866] Updated weights for policy 1, policy_version 77720 (0.0008) -[2023-10-15 17:52:53,156][52833] Updated weights for policy 0, policy_version 77480 (0.0008) -[2023-10-15 17:52:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 158924800. Throughput: 0: 1809.6, 1: 1794.2. Samples: 39741920. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:52:53,442][51532] Avg episode reward: [(0, '65.440'), (1, '74.240')] -[2023-10-15 17:52:53,526][52833] Updated weights for policy 0, policy_version 77490 (0.0008) -[2023-10-15 17:52:53,893][52833] Updated weights for policy 0, policy_version 77500 (0.0009) -[2023-10-15 17:52:54,606][52866] Updated weights for policy 1, policy_version 77730 (0.0010) -[2023-10-15 17:52:54,983][52866] Updated weights for policy 1, policy_version 77740 (0.0010) -[2023-10-15 17:52:55,346][52866] Updated weights for policy 1, policy_version 77750 (0.0010) -[2023-10-15 17:52:55,724][52866] Updated weights for policy 1, policy_version 77760 (0.0010) -[2023-10-15 17:52:57,598][52833] Updated weights for policy 0, policy_version 77510 (0.0009) -[2023-10-15 17:52:57,963][52833] Updated weights for policy 0, policy_version 77520 (0.0008) -[2023-10-15 17:52:58,337][52833] Updated weights for policy 0, policy_version 77530 (0.0007) -[2023-10-15 17:52:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 158990336. Throughput: 0: 1817.6, 1: 1789.9. Samples: 39764138. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:52:58,442][51532] Avg episode reward: [(0, '64.080'), (1, '74.060')] -[2023-10-15 17:52:59,373][52866] Updated weights for policy 1, policy_version 77770 (0.0008) -[2023-10-15 17:52:59,736][52866] Updated weights for policy 1, policy_version 77780 (0.0010) -[2023-10-15 17:53:00,106][52866] Updated weights for policy 1, policy_version 77790 (0.0010) -[2023-10-15 17:53:02,138][52833] Updated weights for policy 0, policy_version 77540 (0.0007) -[2023-10-15 17:53:02,521][52833] Updated weights for policy 0, policy_version 77550 (0.0009) -[2023-10-15 17:53:02,892][52833] Updated weights for policy 0, policy_version 77560 (0.0009) -[2023-10-15 17:53:03,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 159088640. Throughput: 0: 1814.0, 1: 1788.3. Samples: 39774626. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:53:03,442][51532] Avg episode reward: [(0, '63.820'), (1, '71.360')] -[2023-10-15 17:53:03,892][52866] Updated weights for policy 1, policy_version 77800 (0.0009) -[2023-10-15 17:53:04,263][52866] Updated weights for policy 1, policy_version 77810 (0.0008) -[2023-10-15 17:53:04,633][52866] Updated weights for policy 1, policy_version 77820 (0.0009) -[2023-10-15 17:53:06,618][52833] Updated weights for policy 0, policy_version 77570 (0.0009) -[2023-10-15 17:53:06,991][52833] Updated weights for policy 0, policy_version 77580 (0.0010) -[2023-10-15 17:53:07,351][52833] Updated weights for policy 0, policy_version 77590 (0.0007) -[2023-10-15 17:53:07,721][52833] Updated weights for policy 0, policy_version 77600 (0.0008) -[2023-10-15 17:53:08,313][52866] Updated weights for policy 1, policy_version 77830 (0.0009) -[2023-10-15 17:53:08,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 159154176. Throughput: 0: 1814.1, 1: 1784.7. Samples: 39796430. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:53:08,442][51532] Avg episode reward: [(0, '64.110'), (1, '70.460')] -[2023-10-15 17:53:08,677][52866] Updated weights for policy 1, policy_version 77840 (0.0009) -[2023-10-15 17:53:09,045][52866] Updated weights for policy 1, policy_version 77850 (0.0009) -[2023-10-15 17:53:11,406][52833] Updated weights for policy 0, policy_version 77610 (0.0008) -[2023-10-15 17:53:11,772][52833] Updated weights for policy 0, policy_version 77620 (0.0007) -[2023-10-15 17:53:12,146][52833] Updated weights for policy 0, policy_version 77630 (0.0007) -[2023-10-15 17:53:12,820][52866] Updated weights for policy 1, policy_version 77860 (0.0009) -[2023-10-15 17:53:13,179][52866] Updated weights for policy 1, policy_version 77870 (0.0008) -[2023-10-15 17:53:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 159219712. Throughput: 0: 1807.0, 1: 1803.8. Samples: 39817622. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:53:13,441][51532] Avg episode reward: [(0, '66.510'), (1, '72.750')] -[2023-10-15 17:53:13,553][52866] Updated weights for policy 1, policy_version 77880 (0.0007) -[2023-10-15 17:53:15,892][52833] Updated weights for policy 0, policy_version 77640 (0.0007) -[2023-10-15 17:53:16,258][52833] Updated weights for policy 0, policy_version 77650 (0.0008) -[2023-10-15 17:53:16,633][52833] Updated weights for policy 0, policy_version 77660 (0.0009) -[2023-10-15 17:53:17,423][52866] Updated weights for policy 1, policy_version 77890 (0.0010) -[2023-10-15 17:53:17,793][52866] Updated weights for policy 1, policy_version 77900 (0.0007) -[2023-10-15 17:53:18,161][52866] Updated weights for policy 1, policy_version 77910 (0.0007) -[2023-10-15 17:53:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 159285248. Throughput: 0: 1808.9, 1: 1789.1. Samples: 39828846. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:53:18,441][51532] Avg episode reward: [(0, '66.990'), (1, '70.810')] -[2023-10-15 17:53:18,524][52866] Updated weights for policy 1, policy_version 77920 (0.0008) -[2023-10-15 17:53:20,191][52833] Updated weights for policy 0, policy_version 77670 (0.0008) -[2023-10-15 17:53:20,574][52833] Updated weights for policy 0, policy_version 77680 (0.0009) -[2023-10-15 17:53:20,937][52833] Updated weights for policy 0, policy_version 77690 (0.0009) -[2023-10-15 17:53:22,311][52866] Updated weights for policy 1, policy_version 77930 (0.0007) -[2023-10-15 17:53:22,687][52866] Updated weights for policy 1, policy_version 77940 (0.0008) -[2023-10-15 17:53:23,047][52866] Updated weights for policy 1, policy_version 77950 (0.0011) -[2023-10-15 17:53:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 159383552. Throughput: 0: 1805.0, 1: 1812.2. Samples: 39850428. Policy #0 lag: (min: 2.0, avg: 2.9, max: 22.0) -[2023-10-15 17:53:23,441][51532] Avg episode reward: [(0, '67.350'), (1, '75.010')] -[2023-10-15 17:53:24,764][52833] Updated weights for policy 0, policy_version 77700 (0.0007) -[2023-10-15 17:53:25,118][52833] Updated weights for policy 0, policy_version 77710 (0.0009) -[2023-10-15 17:53:25,491][52833] Updated weights for policy 0, policy_version 77720 (0.0008) -[2023-10-15 17:53:26,854][52866] Updated weights for policy 1, policy_version 77960 (0.0010) -[2023-10-15 17:53:27,225][52866] Updated weights for policy 1, policy_version 77970 (0.0009) -[2023-10-15 17:53:27,594][52866] Updated weights for policy 1, policy_version 77980 (0.0008) -[2023-10-15 17:53:28,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 159449088. Throughput: 0: 1805.7, 1: 1796.7. Samples: 39871586. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:53:28,442][51532] Avg episode reward: [(0, '68.080'), (1, '74.660')] -[2023-10-15 17:53:29,275][52833] Updated weights for policy 0, policy_version 77730 (0.0008) -[2023-10-15 17:53:29,636][52833] Updated weights for policy 0, policy_version 77740 (0.0007) -[2023-10-15 17:53:30,017][52833] Updated weights for policy 0, policy_version 77750 (0.0009) -[2023-10-15 17:53:30,383][52833] Updated weights for policy 0, policy_version 77760 (0.0010) -[2023-10-15 17:53:31,182][52866] Updated weights for policy 1, policy_version 77990 (0.0009) -[2023-10-15 17:53:31,552][52866] Updated weights for policy 1, policy_version 78000 (0.0007) -[2023-10-15 17:53:31,911][52866] Updated weights for policy 1, policy_version 78010 (0.0009) -[2023-10-15 17:53:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 159514624. Throughput: 0: 1804.4, 1: 1814.0. Samples: 39882908. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:53:33,442][51532] Avg episode reward: [(0, '68.470'), (1, '74.700')] -[2023-10-15 17:53:34,099][52833] Updated weights for policy 0, policy_version 77770 (0.0010) -[2023-10-15 17:53:34,465][52833] Updated weights for policy 0, policy_version 77780 (0.0009) -[2023-10-15 17:53:34,837][52833] Updated weights for policy 0, policy_version 77790 (0.0010) -[2023-10-15 17:53:35,582][52866] Updated weights for policy 1, policy_version 78020 (0.0010) -[2023-10-15 17:53:35,946][52866] Updated weights for policy 1, policy_version 78030 (0.0008) -[2023-10-15 17:53:36,305][52866] Updated weights for policy 1, policy_version 78040 (0.0010) -[2023-10-15 17:53:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 159580160. Throughput: 0: 1801.1, 1: 1800.9. Samples: 39904012. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:53:38,441][51532] Avg episode reward: [(0, '66.430'), (1, '71.670')] -[2023-10-15 17:53:38,673][52833] Updated weights for policy 0, policy_version 77800 (0.0009) -[2023-10-15 17:53:39,048][52833] Updated weights for policy 0, policy_version 77810 (0.0010) -[2023-10-15 17:53:39,413][52833] Updated weights for policy 0, policy_version 77820 (0.0010) -[2023-10-15 17:53:39,935][52866] Updated weights for policy 1, policy_version 78050 (0.0008) -[2023-10-15 17:53:40,305][52866] Updated weights for policy 1, policy_version 78060 (0.0012) -[2023-10-15 17:53:40,668][52866] Updated weights for policy 1, policy_version 78070 (0.0010) -[2023-10-15 17:53:41,035][52866] Updated weights for policy 1, policy_version 78080 (0.0009) -[2023-10-15 17:53:43,161][52833] Updated weights for policy 0, policy_version 77830 (0.0010) -[2023-10-15 17:53:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 159645696. Throughput: 0: 1809.0, 1: 1801.6. Samples: 39926616. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:53:43,441][51532] Avg episode reward: [(0, '66.780'), (1, '67.450')] -[2023-10-15 17:53:43,448][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000078080_79953920.pth... -[2023-10-15 17:53:43,484][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000076416_78249984.pth -[2023-10-15 17:53:43,535][52833] Updated weights for policy 0, policy_version 77840 (0.0008) -[2023-10-15 17:53:43,908][52833] Updated weights for policy 0, policy_version 77850 (0.0009) -[2023-10-15 17:53:44,126][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000077856_79724544.pth... -[2023-10-15 17:53:44,166][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000076160_77987840.pth -[2023-10-15 17:53:44,659][52866] Updated weights for policy 1, policy_version 78090 (0.0009) -[2023-10-15 17:53:45,038][52866] Updated weights for policy 1, policy_version 78100 (0.0010) -[2023-10-15 17:53:45,397][52866] Updated weights for policy 1, policy_version 78110 (0.0010) -[2023-10-15 17:53:47,695][52833] Updated weights for policy 0, policy_version 77860 (0.0009) -[2023-10-15 17:53:48,095][52833] Updated weights for policy 0, policy_version 77870 (0.0010) -[2023-10-15 17:53:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 159711232. Throughput: 0: 1797.6, 1: 1803.4. Samples: 39936672. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:53:48,441][51532] Avg episode reward: [(0, '65.790'), (1, '68.860')] -[2023-10-15 17:53:48,464][52833] Updated weights for policy 0, policy_version 77880 (0.0009) -[2023-10-15 17:53:49,196][52866] Updated weights for policy 1, policy_version 78120 (0.0008) -[2023-10-15 17:53:49,568][52866] Updated weights for policy 1, policy_version 78130 (0.0008) -[2023-10-15 17:53:49,936][52866] Updated weights for policy 1, policy_version 78140 (0.0008) -[2023-10-15 17:53:52,185][52833] Updated weights for policy 0, policy_version 77890 (0.0010) -[2023-10-15 17:53:52,560][52833] Updated weights for policy 0, policy_version 77900 (0.0008) -[2023-10-15 17:53:52,926][52833] Updated weights for policy 0, policy_version 77910 (0.0009) -[2023-10-15 17:53:53,298][52833] Updated weights for policy 0, policy_version 77920 (0.0009) -[2023-10-15 17:53:53,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 159809536. Throughput: 0: 1807.1, 1: 1794.5. Samples: 39958502. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:53:53,442][51532] Avg episode reward: [(0, '66.240'), (1, '68.670')] -[2023-10-15 17:53:53,782][52866] Updated weights for policy 1, policy_version 78150 (0.0010) -[2023-10-15 17:53:54,147][52866] Updated weights for policy 1, policy_version 78160 (0.0010) -[2023-10-15 17:53:54,521][52866] Updated weights for policy 1, policy_version 78170 (0.0012) -[2023-10-15 17:53:56,959][52833] Updated weights for policy 0, policy_version 77930 (0.0010) -[2023-10-15 17:53:57,332][52833] Updated weights for policy 0, policy_version 77940 (0.0007) -[2023-10-15 17:53:57,702][52833] Updated weights for policy 0, policy_version 77950 (0.0010) -[2023-10-15 17:53:58,358][52866] Updated weights for policy 1, policy_version 78180 (0.0008) -[2023-10-15 17:53:58,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 159875072. Throughput: 0: 1792.9, 1: 1804.2. Samples: 39979492. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:53:58,442][51532] Avg episode reward: [(0, '68.200'), (1, '69.770')] -[2023-10-15 17:53:58,740][52866] Updated weights for policy 1, policy_version 78190 (0.0009) -[2023-10-15 17:53:59,110][52866] Updated weights for policy 1, policy_version 78200 (0.0008) -[2023-10-15 17:54:01,534][52833] Updated weights for policy 0, policy_version 77960 (0.0008) -[2023-10-15 17:54:01,901][52833] Updated weights for policy 0, policy_version 77970 (0.0007) -[2023-10-15 17:54:02,267][52833] Updated weights for policy 0, policy_version 77980 (0.0007) -[2023-10-15 17:54:02,899][52866] Updated weights for policy 1, policy_version 78210 (0.0008) -[2023-10-15 17:54:03,268][52866] Updated weights for policy 1, policy_version 78220 (0.0007) -[2023-10-15 17:54:03,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 159940608. Throughput: 0: 1801.8, 1: 1794.6. Samples: 39990686. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:54:03,441][51532] Avg episode reward: [(0, '69.520'), (1, '70.540')] -[2023-10-15 17:54:03,633][52866] Updated weights for policy 1, policy_version 78230 (0.0008) -[2023-10-15 17:54:03,997][52866] Updated weights for policy 1, policy_version 78240 (0.0009) -[2023-10-15 17:54:06,020][52833] Updated weights for policy 0, policy_version 77990 (0.0010) -[2023-10-15 17:54:06,397][52833] Updated weights for policy 0, policy_version 78000 (0.0010) -[2023-10-15 17:54:06,764][52833] Updated weights for policy 0, policy_version 78010 (0.0009) -[2023-10-15 17:54:07,669][52866] Updated weights for policy 1, policy_version 78250 (0.0009) -[2023-10-15 17:54:08,029][52866] Updated weights for policy 1, policy_version 78260 (0.0008) -[2023-10-15 17:54:08,391][52866] Updated weights for policy 1, policy_version 78270 (0.0007) -[2023-10-15 17:54:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 160006144. Throughput: 0: 1794.8, 1: 1794.0. Samples: 40011926. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:54:08,441][51532] Avg episode reward: [(0, '69.890'), (1, '71.100')] -[2023-10-15 17:54:10,285][52833] Updated weights for policy 0, policy_version 78020 (0.0010) -[2023-10-15 17:54:10,650][52833] Updated weights for policy 0, policy_version 78030 (0.0008) -[2023-10-15 17:54:11,030][52833] Updated weights for policy 0, policy_version 78040 (0.0008) -[2023-10-15 17:54:12,235][52866] Updated weights for policy 1, policy_version 78280 (0.0007) -[2023-10-15 17:54:12,603][52866] Updated weights for policy 1, policy_version 78290 (0.0008) -[2023-10-15 17:54:12,961][52866] Updated weights for policy 1, policy_version 78300 (0.0007) -[2023-10-15 17:54:13,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 160104448. Throughput: 0: 1795.7, 1: 1800.4. Samples: 40033412. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-15 17:54:13,442][51532] Avg episode reward: [(0, '65.850'), (1, '69.600')] -[2023-10-15 17:54:14,749][52833] Updated weights for policy 0, policy_version 78050 (0.0008) -[2023-10-15 17:54:15,119][52833] Updated weights for policy 0, policy_version 78060 (0.0008) -[2023-10-15 17:54:15,492][52833] Updated weights for policy 0, policy_version 78070 (0.0010) -[2023-10-15 17:54:15,860][52833] Updated weights for policy 0, policy_version 78080 (0.0011) -[2023-10-15 17:54:16,706][52866] Updated weights for policy 1, policy_version 78310 (0.0008) -[2023-10-15 17:54:17,069][52866] Updated weights for policy 1, policy_version 78320 (0.0007) -[2023-10-15 17:54:17,447][52866] Updated weights for policy 1, policy_version 78330 (0.0008) -[2023-10-15 17:54:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 160169984. Throughput: 0: 1798.4, 1: 1792.6. Samples: 40044502. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:18,442][51532] Avg episode reward: [(0, '67.220'), (1, '66.160')] -[2023-10-15 17:54:19,587][52833] Updated weights for policy 0, policy_version 78090 (0.0008) -[2023-10-15 17:54:19,962][52833] Updated weights for policy 0, policy_version 78100 (0.0007) -[2023-10-15 17:54:20,332][52833] Updated weights for policy 0, policy_version 78110 (0.0008) -[2023-10-15 17:54:21,025][52866] Updated weights for policy 1, policy_version 78340 (0.0009) -[2023-10-15 17:54:21,393][52866] Updated weights for policy 1, policy_version 78350 (0.0009) -[2023-10-15 17:54:21,754][52866] Updated weights for policy 1, policy_version 78360 (0.0007) -[2023-10-15 17:54:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 160235520. Throughput: 0: 1795.9, 1: 1800.6. Samples: 40065854. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:23,442][51532] Avg episode reward: [(0, '66.350'), (1, '62.890')] -[2023-10-15 17:54:24,211][52833] Updated weights for policy 0, policy_version 78120 (0.0008) -[2023-10-15 17:54:24,588][52833] Updated weights for policy 0, policy_version 78130 (0.0008) -[2023-10-15 17:54:24,955][52833] Updated weights for policy 0, policy_version 78140 (0.0009) -[2023-10-15 17:54:25,534][52866] Updated weights for policy 1, policy_version 78370 (0.0009) -[2023-10-15 17:54:25,907][52866] Updated weights for policy 1, policy_version 78380 (0.0007) -[2023-10-15 17:54:26,259][52866] Updated weights for policy 1, policy_version 78390 (0.0009) -[2023-10-15 17:54:26,627][52866] Updated weights for policy 1, policy_version 78400 (0.0007) -[2023-10-15 17:54:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 160301056. Throughput: 0: 1797.6, 1: 1792.1. Samples: 40088154. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:28,442][51532] Avg episode reward: [(0, '66.410'), (1, '63.700')] -[2023-10-15 17:54:28,544][52833] Updated weights for policy 0, policy_version 78150 (0.0009) -[2023-10-15 17:54:28,919][52833] Updated weights for policy 0, policy_version 78160 (0.0007) -[2023-10-15 17:54:29,289][52833] Updated weights for policy 0, policy_version 78170 (0.0008) -[2023-10-15 17:54:30,360][52866] Updated weights for policy 1, policy_version 78410 (0.0010) -[2023-10-15 17:54:30,719][52866] Updated weights for policy 1, policy_version 78420 (0.0009) -[2023-10-15 17:54:31,093][52866] Updated weights for policy 1, policy_version 78430 (0.0008) -[2023-10-15 17:54:33,251][52833] Updated weights for policy 0, policy_version 78180 (0.0010) -[2023-10-15 17:54:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 160366592. Throughput: 0: 1791.6, 1: 1800.6. Samples: 40098322. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:33,441][51532] Avg episode reward: [(0, '63.350'), (1, '62.190')] -[2023-10-15 17:54:33,650][52833] Updated weights for policy 0, policy_version 78190 (0.0010) -[2023-10-15 17:54:34,010][52833] Updated weights for policy 0, policy_version 78200 (0.0009) -[2023-10-15 17:54:34,738][52866] Updated weights for policy 1, policy_version 78440 (0.0008) -[2023-10-15 17:54:35,102][52866] Updated weights for policy 1, policy_version 78450 (0.0010) -[2023-10-15 17:54:35,474][52866] Updated weights for policy 1, policy_version 78460 (0.0008) -[2023-10-15 17:54:37,683][52833] Updated weights for policy 0, policy_version 78210 (0.0008) -[2023-10-15 17:54:38,056][52833] Updated weights for policy 0, policy_version 78220 (0.0010) -[2023-10-15 17:54:38,427][52833] Updated weights for policy 0, policy_version 78230 (0.0007) -[2023-10-15 17:54:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 160432128. Throughput: 0: 1790.4, 1: 1803.9. Samples: 40120244. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:38,441][51532] Avg episode reward: [(0, '62.240'), (1, '63.600')] -[2023-10-15 17:54:38,802][52833] Updated weights for policy 0, policy_version 78240 (0.0008) -[2023-10-15 17:54:39,330][52866] Updated weights for policy 1, policy_version 78470 (0.0011) -[2023-10-15 17:54:39,708][52866] Updated weights for policy 1, policy_version 78480 (0.0009) -[2023-10-15 17:54:40,080][52866] Updated weights for policy 1, policy_version 78490 (0.0009) -[2023-10-15 17:54:42,515][52833] Updated weights for policy 0, policy_version 78250 (0.0008) -[2023-10-15 17:54:42,874][52833] Updated weights for policy 0, policy_version 78260 (0.0012) -[2023-10-15 17:54:43,242][52833] Updated weights for policy 0, policy_version 78270 (0.0008) -[2023-10-15 17:54:43,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 160530432. Throughput: 0: 1805.3, 1: 1800.4. Samples: 40141750. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:43,442][51532] Avg episode reward: [(0, '60.130'), (1, '68.290')] -[2023-10-15 17:54:43,985][52866] Updated weights for policy 1, policy_version 78500 (0.0010) -[2023-10-15 17:54:44,378][52866] Updated weights for policy 1, policy_version 78510 (0.0007) -[2023-10-15 17:54:44,747][52866] Updated weights for policy 1, policy_version 78520 (0.0009) -[2023-10-15 17:54:46,972][52833] Updated weights for policy 0, policy_version 78280 (0.0011) -[2023-10-15 17:54:47,341][52833] Updated weights for policy 0, policy_version 78290 (0.0009) -[2023-10-15 17:54:47,712][52833] Updated weights for policy 0, policy_version 78300 (0.0008) -[2023-10-15 17:54:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 160595968. Throughput: 0: 1795.2, 1: 1801.8. Samples: 40152550. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:48,441][51532] Avg episode reward: [(0, '63.310'), (1, '67.120')] -[2023-10-15 17:54:48,464][52866] Updated weights for policy 1, policy_version 78530 (0.0009) -[2023-10-15 17:54:48,822][52866] Updated weights for policy 1, policy_version 78540 (0.0007) -[2023-10-15 17:54:49,185][52866] Updated weights for policy 1, policy_version 78550 (0.0008) -[2023-10-15 17:54:49,562][52866] Updated weights for policy 1, policy_version 78560 (0.0011) -[2023-10-15 17:54:51,457][52833] Updated weights for policy 0, policy_version 78310 (0.0009) -[2023-10-15 17:54:51,821][52833] Updated weights for policy 0, policy_version 78320 (0.0007) -[2023-10-15 17:54:52,186][52833] Updated weights for policy 0, policy_version 78330 (0.0008) -[2023-10-15 17:54:53,346][52866] Updated weights for policy 1, policy_version 78570 (0.0008) -[2023-10-15 17:54:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 160661504. Throughput: 0: 1809.2, 1: 1798.9. Samples: 40174292. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:53,442][51532] Avg episode reward: [(0, '61.550'), (1, '67.510')] -[2023-10-15 17:54:53,717][52866] Updated weights for policy 1, policy_version 78580 (0.0007) -[2023-10-15 17:54:54,086][52866] Updated weights for policy 1, policy_version 78590 (0.0008) -[2023-10-15 17:54:55,988][52833] Updated weights for policy 0, policy_version 78340 (0.0009) -[2023-10-15 17:54:56,355][52833] Updated weights for policy 0, policy_version 78350 (0.0010) -[2023-10-15 17:54:56,722][52833] Updated weights for policy 0, policy_version 78360 (0.0008) -[2023-10-15 17:54:57,820][52866] Updated weights for policy 1, policy_version 78600 (0.0008) -[2023-10-15 17:54:58,183][52866] Updated weights for policy 1, policy_version 78610 (0.0007) -[2023-10-15 17:54:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 160727040. Throughput: 0: 1787.0, 1: 1809.7. Samples: 40195262. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:54:58,442][51532] Avg episode reward: [(0, '59.500'), (1, '65.690')] -[2023-10-15 17:54:58,552][52866] Updated weights for policy 1, policy_version 78620 (0.0009) -[2023-10-15 17:55:00,381][52833] Updated weights for policy 0, policy_version 78370 (0.0007) -[2023-10-15 17:55:00,746][52833] Updated weights for policy 0, policy_version 78380 (0.0007) -[2023-10-15 17:55:01,120][52833] Updated weights for policy 0, policy_version 78390 (0.0008) -[2023-10-15 17:55:01,487][52833] Updated weights for policy 0, policy_version 78400 (0.0008) -[2023-10-15 17:55:02,300][52866] Updated weights for policy 1, policy_version 78630 (0.0010) -[2023-10-15 17:55:02,658][52866] Updated weights for policy 1, policy_version 78640 (0.0010) -[2023-10-15 17:55:03,031][52866] Updated weights for policy 1, policy_version 78650 (0.0011) -[2023-10-15 17:55:03,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 160825344. Throughput: 0: 1807.3, 1: 1796.1. Samples: 40206658. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) -[2023-10-15 17:55:03,442][51532] Avg episode reward: [(0, '60.110'), (1, '66.330')] -[2023-10-15 17:55:05,315][52833] Updated weights for policy 0, policy_version 78410 (0.0010) -[2023-10-15 17:55:05,688][52833] Updated weights for policy 0, policy_version 78420 (0.0008) -[2023-10-15 17:55:06,058][52833] Updated weights for policy 0, policy_version 78430 (0.0008) -[2023-10-15 17:55:06,594][52866] Updated weights for policy 1, policy_version 78660 (0.0007) -[2023-10-15 17:55:06,958][52866] Updated weights for policy 1, policy_version 78670 (0.0007) -[2023-10-15 17:55:07,319][52866] Updated weights for policy 1, policy_version 78680 (0.0010) -[2023-10-15 17:55:08,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 160890880. Throughput: 0: 1791.1, 1: 1812.3. Samples: 40228006. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:08,442][51532] Avg episode reward: [(0, '57.580'), (1, '67.930')] -[2023-10-15 17:55:09,690][52833] Updated weights for policy 0, policy_version 78440 (0.0008) -[2023-10-15 17:55:10,056][52833] Updated weights for policy 0, policy_version 78450 (0.0008) -[2023-10-15 17:55:10,420][52833] Updated weights for policy 0, policy_version 78460 (0.0007) -[2023-10-15 17:55:11,057][52866] Updated weights for policy 1, policy_version 78690 (0.0010) -[2023-10-15 17:55:11,425][52866] Updated weights for policy 1, policy_version 78700 (0.0010) -[2023-10-15 17:55:11,795][52866] Updated weights for policy 1, policy_version 78710 (0.0011) -[2023-10-15 17:55:12,169][52866] Updated weights for policy 1, policy_version 78720 (0.0009) -[2023-10-15 17:55:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 160956416. Throughput: 0: 1790.8, 1: 1802.2. Samples: 40249842. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:13,442][51532] Avg episode reward: [(0, '55.380'), (1, '66.640')] -[2023-10-15 17:55:14,200][52833] Updated weights for policy 0, policy_version 78470 (0.0008) -[2023-10-15 17:55:14,571][52833] Updated weights for policy 0, policy_version 78480 (0.0007) -[2023-10-15 17:55:14,935][52833] Updated weights for policy 0, policy_version 78490 (0.0007) -[2023-10-15 17:55:15,769][52866] Updated weights for policy 1, policy_version 78730 (0.0008) -[2023-10-15 17:55:16,126][52866] Updated weights for policy 1, policy_version 78740 (0.0009) -[2023-10-15 17:55:16,485][52866] Updated weights for policy 1, policy_version 78750 (0.0009) -[2023-10-15 17:55:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161021952. Throughput: 0: 1796.6, 1: 1813.2. Samples: 40260764. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:18,441][51532] Avg episode reward: [(0, '53.600'), (1, '68.350')] -[2023-10-15 17:55:18,652][52833] Updated weights for policy 0, policy_version 78500 (0.0008) -[2023-10-15 17:55:19,025][52833] Updated weights for policy 0, policy_version 78510 (0.0009) -[2023-10-15 17:55:19,403][52833] Updated weights for policy 0, policy_version 78520 (0.0010) -[2023-10-15 17:55:20,053][52866] Updated weights for policy 1, policy_version 78760 (0.0008) -[2023-10-15 17:55:20,417][52866] Updated weights for policy 1, policy_version 78770 (0.0007) -[2023-10-15 17:55:20,782][52866] Updated weights for policy 1, policy_version 78780 (0.0009) -[2023-10-15 17:55:23,099][52833] Updated weights for policy 0, policy_version 78530 (0.0011) -[2023-10-15 17:55:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161087488. Throughput: 0: 1807.1, 1: 1800.7. Samples: 40282596. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:23,441][51532] Avg episode reward: [(0, '55.240'), (1, '68.050')] -[2023-10-15 17:55:23,466][52833] Updated weights for policy 0, policy_version 78540 (0.0010) -[2023-10-15 17:55:23,831][52833] Updated weights for policy 0, policy_version 78550 (0.0009) -[2023-10-15 17:55:24,201][52833] Updated weights for policy 0, policy_version 78560 (0.0008) -[2023-10-15 17:55:24,692][52866] Updated weights for policy 1, policy_version 78790 (0.0010) -[2023-10-15 17:55:25,068][52866] Updated weights for policy 1, policy_version 78800 (0.0009) -[2023-10-15 17:55:25,431][52866] Updated weights for policy 1, policy_version 78810 (0.0009) -[2023-10-15 17:55:27,875][52833] Updated weights for policy 0, policy_version 78570 (0.0007) -[2023-10-15 17:55:28,242][52833] Updated weights for policy 0, policy_version 78580 (0.0007) -[2023-10-15 17:55:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161153024. Throughput: 0: 1823.0, 1: 1799.5. Samples: 40304764. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:28,441][51532] Avg episode reward: [(0, '62.650'), (1, '66.630')] -[2023-10-15 17:55:28,609][52833] Updated weights for policy 0, policy_version 78590 (0.0008) -[2023-10-15 17:55:29,394][52866] Updated weights for policy 1, policy_version 78820 (0.0010) -[2023-10-15 17:55:29,785][52866] Updated weights for policy 1, policy_version 78830 (0.0009) -[2023-10-15 17:55:30,149][52866] Updated weights for policy 1, policy_version 78840 (0.0008) -[2023-10-15 17:55:32,460][52833] Updated weights for policy 0, policy_version 78600 (0.0007) -[2023-10-15 17:55:32,824][52833] Updated weights for policy 0, policy_version 78610 (0.0007) -[2023-10-15 17:55:33,193][52833] Updated weights for policy 0, policy_version 78620 (0.0007) -[2023-10-15 17:55:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 161251328. Throughput: 0: 1810.4, 1: 1796.5. Samples: 40314864. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:33,441][51532] Avg episode reward: [(0, '60.590'), (1, '66.710')] -[2023-10-15 17:55:33,924][52866] Updated weights for policy 1, policy_version 78850 (0.0009) -[2023-10-15 17:55:34,288][52866] Updated weights for policy 1, policy_version 78860 (0.0008) -[2023-10-15 17:55:34,656][52866] Updated weights for policy 1, policy_version 78870 (0.0008) -[2023-10-15 17:55:35,022][52866] Updated weights for policy 1, policy_version 78880 (0.0009) -[2023-10-15 17:55:36,785][52833] Updated weights for policy 0, policy_version 78630 (0.0007) -[2023-10-15 17:55:37,150][52833] Updated weights for policy 0, policy_version 78640 (0.0008) -[2023-10-15 17:55:37,520][52833] Updated weights for policy 0, policy_version 78650 (0.0008) -[2023-10-15 17:55:38,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 161316864. Throughput: 0: 1811.9, 1: 1800.3. Samples: 40336840. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:38,441][51532] Avg episode reward: [(0, '58.590'), (1, '63.370')] -[2023-10-15 17:55:38,707][52866] Updated weights for policy 1, policy_version 78890 (0.0012) -[2023-10-15 17:55:39,079][52866] Updated weights for policy 1, policy_version 78900 (0.0011) -[2023-10-15 17:55:39,442][52866] Updated weights for policy 1, policy_version 78910 (0.0010) -[2023-10-15 17:55:41,372][52833] Updated weights for policy 0, policy_version 78660 (0.0009) -[2023-10-15 17:55:41,740][52833] Updated weights for policy 0, policy_version 78670 (0.0007) -[2023-10-15 17:55:42,099][52833] Updated weights for policy 0, policy_version 78680 (0.0007) -[2023-10-15 17:55:43,042][52866] Updated weights for policy 1, policy_version 78920 (0.0007) -[2023-10-15 17:55:43,404][52866] Updated weights for policy 1, policy_version 78930 (0.0007) -[2023-10-15 17:55:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 161382400. Throughput: 0: 1805.7, 1: 1812.9. Samples: 40358100. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:43,441][51532] Avg episode reward: [(0, '60.500'), (1, '61.180')] -[2023-10-15 17:55:43,449][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000078688_80576512.pth... -[2023-10-15 17:55:43,486][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000076992_78839808.pth -[2023-10-15 17:55:43,767][52866] Updated weights for policy 1, policy_version 78940 (0.0008) -[2023-10-15 17:55:43,915][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000078944_80838656.pth... -[2023-10-15 17:55:43,954][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000077248_79101952.pth -[2023-10-15 17:55:45,759][52833] Updated weights for policy 0, policy_version 78690 (0.0010) -[2023-10-15 17:55:46,137][52833] Updated weights for policy 0, policy_version 78700 (0.0008) -[2023-10-15 17:55:46,500][52833] Updated weights for policy 0, policy_version 78710 (0.0011) -[2023-10-15 17:55:46,873][52833] Updated weights for policy 0, policy_version 78720 (0.0010) -[2023-10-15 17:55:47,398][52866] Updated weights for policy 1, policy_version 78950 (0.0009) -[2023-10-15 17:55:47,762][52866] Updated weights for policy 1, policy_version 78960 (0.0011) -[2023-10-15 17:55:48,133][52866] Updated weights for policy 1, policy_version 78970 (0.0011) -[2023-10-15 17:55:48,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 161480704. Throughput: 0: 1811.4, 1: 1805.6. Samples: 40369424. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:48,441][51532] Avg episode reward: [(0, '62.220'), (1, '60.330')] -[2023-10-15 17:55:50,655][52833] Updated weights for policy 0, policy_version 78730 (0.0008) -[2023-10-15 17:55:51,029][52833] Updated weights for policy 0, policy_version 78740 (0.0008) -[2023-10-15 17:55:51,388][52833] Updated weights for policy 0, policy_version 78750 (0.0007) -[2023-10-15 17:55:51,958][52866] Updated weights for policy 1, policy_version 78980 (0.0010) -[2023-10-15 17:55:52,334][52866] Updated weights for policy 1, policy_version 78990 (0.0009) -[2023-10-15 17:55:52,690][52866] Updated weights for policy 1, policy_version 79000 (0.0009) -[2023-10-15 17:55:53,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 161546240. Throughput: 0: 1802.4, 1: 1809.5. Samples: 40390546. Policy #0 lag: (min: 18.0, avg: 19.1, max: 41.0) -[2023-10-15 17:55:53,442][51532] Avg episode reward: [(0, '64.610'), (1, '62.670')] -[2023-10-15 17:55:55,238][52833] Updated weights for policy 0, policy_version 78760 (0.0007) -[2023-10-15 17:55:55,607][52833] Updated weights for policy 0, policy_version 78770 (0.0009) -[2023-10-15 17:55:55,976][52833] Updated weights for policy 0, policy_version 78780 (0.0010) -[2023-10-15 17:55:56,504][52866] Updated weights for policy 1, policy_version 79010 (0.0010) -[2023-10-15 17:55:56,872][52866] Updated weights for policy 1, policy_version 79020 (0.0009) -[2023-10-15 17:55:57,247][52866] Updated weights for policy 1, policy_version 79030 (0.0011) -[2023-10-15 17:55:57,611][52866] Updated weights for policy 1, policy_version 79040 (0.0010) -[2023-10-15 17:55:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 161611776. Throughput: 0: 1804.7, 1: 1795.8. Samples: 40411864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:55:58,441][51532] Avg episode reward: [(0, '63.860'), (1, '61.550')] -[2023-10-15 17:55:59,750][52833] Updated weights for policy 0, policy_version 78790 (0.0008) -[2023-10-15 17:56:00,125][52833] Updated weights for policy 0, policy_version 78800 (0.0008) -[2023-10-15 17:56:00,495][52833] Updated weights for policy 0, policy_version 78810 (0.0007) -[2023-10-15 17:56:01,343][52866] Updated weights for policy 1, policy_version 79050 (0.0008) -[2023-10-15 17:56:01,713][52866] Updated weights for policy 1, policy_version 79060 (0.0009) -[2023-10-15 17:56:02,086][52866] Updated weights for policy 1, policy_version 79070 (0.0009) -[2023-10-15 17:56:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161677312. Throughput: 0: 1800.4, 1: 1809.1. Samples: 40423190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:03,442][51532] Avg episode reward: [(0, '64.400'), (1, '61.400')] -[2023-10-15 17:56:04,220][52833] Updated weights for policy 0, policy_version 78820 (0.0009) -[2023-10-15 17:56:04,606][52833] Updated weights for policy 0, policy_version 78830 (0.0008) -[2023-10-15 17:56:04,973][52833] Updated weights for policy 0, policy_version 78840 (0.0008) -[2023-10-15 17:56:05,858][52866] Updated weights for policy 1, policy_version 79080 (0.0008) -[2023-10-15 17:56:06,234][52866] Updated weights for policy 1, policy_version 79090 (0.0008) -[2023-10-15 17:56:06,607][52866] Updated weights for policy 1, policy_version 79100 (0.0008) -[2023-10-15 17:56:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161742848. Throughput: 0: 1800.4, 1: 1795.5. Samples: 40444410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:08,442][51532] Avg episode reward: [(0, '65.990'), (1, '59.270')] -[2023-10-15 17:56:08,556][52833] Updated weights for policy 0, policy_version 78850 (0.0009) -[2023-10-15 17:56:08,924][52833] Updated weights for policy 0, policy_version 78860 (0.0010) -[2023-10-15 17:56:09,295][52833] Updated weights for policy 0, policy_version 78870 (0.0008) -[2023-10-15 17:56:09,659][52833] Updated weights for policy 0, policy_version 78880 (0.0008) -[2023-10-15 17:56:10,442][52866] Updated weights for policy 1, policy_version 79110 (0.0008) -[2023-10-15 17:56:10,806][52866] Updated weights for policy 1, policy_version 79120 (0.0009) -[2023-10-15 17:56:11,175][52866] Updated weights for policy 1, policy_version 79130 (0.0010) -[2023-10-15 17:56:13,418][52833] Updated weights for policy 0, policy_version 78890 (0.0007) -[2023-10-15 17:56:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161808384. Throughput: 0: 1808.3, 1: 1799.9. Samples: 40467134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:13,441][51532] Avg episode reward: [(0, '69.790'), (1, '62.770')] -[2023-10-15 17:56:13,788][52833] Updated weights for policy 0, policy_version 78900 (0.0007) -[2023-10-15 17:56:14,169][52833] Updated weights for policy 0, policy_version 78910 (0.0008) -[2023-10-15 17:56:14,872][52866] Updated weights for policy 1, policy_version 79140 (0.0010) -[2023-10-15 17:56:15,277][52866] Updated weights for policy 1, policy_version 79150 (0.0011) -[2023-10-15 17:56:15,647][52866] Updated weights for policy 1, policy_version 79160 (0.0009) -[2023-10-15 17:56:17,931][52833] Updated weights for policy 0, policy_version 78920 (0.0009) -[2023-10-15 17:56:18,308][52833] Updated weights for policy 0, policy_version 78930 (0.0010) -[2023-10-15 17:56:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 161873920. Throughput: 0: 1798.6, 1: 1802.6. Samples: 40476916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:18,442][51532] Avg episode reward: [(0, '71.070'), (1, '63.330')] -[2023-10-15 17:56:18,676][52833] Updated weights for policy 0, policy_version 78940 (0.0008) -[2023-10-15 17:56:19,361][52866] Updated weights for policy 1, policy_version 79170 (0.0010) -[2023-10-15 17:56:19,719][52866] Updated weights for policy 1, policy_version 79180 (0.0007) -[2023-10-15 17:56:20,088][52866] Updated weights for policy 1, policy_version 79190 (0.0007) -[2023-10-15 17:56:20,449][52866] Updated weights for policy 1, policy_version 79200 (0.0007) -[2023-10-15 17:56:22,373][52833] Updated weights for policy 0, policy_version 78950 (0.0007) -[2023-10-15 17:56:22,744][52833] Updated weights for policy 0, policy_version 78960 (0.0009) -[2023-10-15 17:56:23,107][52833] Updated weights for policy 0, policy_version 78970 (0.0009) -[2023-10-15 17:56:23,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 161972224. Throughput: 0: 1809.0, 1: 1803.9. Samples: 40499420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:23,442][51532] Avg episode reward: [(0, '72.110'), (1, '65.150')] -[2023-10-15 17:56:24,169][52866] Updated weights for policy 1, policy_version 79210 (0.0010) -[2023-10-15 17:56:24,539][52866] Updated weights for policy 1, policy_version 79220 (0.0011) -[2023-10-15 17:56:24,907][52866] Updated weights for policy 1, policy_version 79230 (0.0010) -[2023-10-15 17:56:26,900][52833] Updated weights for policy 0, policy_version 78980 (0.0007) -[2023-10-15 17:56:27,265][52833] Updated weights for policy 0, policy_version 78990 (0.0007) -[2023-10-15 17:56:27,637][52833] Updated weights for policy 0, policy_version 79000 (0.0009) -[2023-10-15 17:56:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 162037760. Throughput: 0: 1804.1, 1: 1810.2. Samples: 40520744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:28,442][51532] Avg episode reward: [(0, '69.060'), (1, '62.900')] -[2023-10-15 17:56:28,541][52866] Updated weights for policy 1, policy_version 79240 (0.0010) -[2023-10-15 17:56:28,917][52866] Updated weights for policy 1, policy_version 79250 (0.0008) -[2023-10-15 17:56:29,278][52866] Updated weights for policy 1, policy_version 79260 (0.0008) -[2023-10-15 17:56:31,210][52833] Updated weights for policy 0, policy_version 79010 (0.0007) -[2023-10-15 17:56:31,569][52833] Updated weights for policy 0, policy_version 79020 (0.0010) -[2023-10-15 17:56:31,938][52833] Updated weights for policy 0, policy_version 79030 (0.0010) -[2023-10-15 17:56:32,306][52833] Updated weights for policy 0, policy_version 79040 (0.0011) -[2023-10-15 17:56:33,046][52866] Updated weights for policy 1, policy_version 79270 (0.0007) -[2023-10-15 17:56:33,410][52866] Updated weights for policy 1, policy_version 79280 (0.0008) -[2023-10-15 17:56:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 162103296. Throughput: 0: 1808.5, 1: 1804.7. Samples: 40532020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:33,441][51532] Avg episode reward: [(0, '69.270'), (1, '65.780')] -[2023-10-15 17:56:33,779][52866] Updated weights for policy 1, policy_version 79290 (0.0008) -[2023-10-15 17:56:35,897][52833] Updated weights for policy 0, policy_version 79050 (0.0009) -[2023-10-15 17:56:36,265][52833] Updated weights for policy 0, policy_version 79060 (0.0009) -[2023-10-15 17:56:36,637][52833] Updated weights for policy 0, policy_version 79070 (0.0009) -[2023-10-15 17:56:37,568][52866] Updated weights for policy 1, policy_version 79300 (0.0009) -[2023-10-15 17:56:37,949][52866] Updated weights for policy 1, policy_version 79310 (0.0009) -[2023-10-15 17:56:38,303][52866] Updated weights for policy 1, policy_version 79320 (0.0011) -[2023-10-15 17:56:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 162168832. Throughput: 0: 1803.6, 1: 1806.5. Samples: 40552998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:38,442][51532] Avg episode reward: [(0, '68.560'), (1, '65.340')] -[2023-10-15 17:56:40,238][52833] Updated weights for policy 0, policy_version 79080 (0.0007) -[2023-10-15 17:56:40,612][52833] Updated weights for policy 0, policy_version 79090 (0.0008) -[2023-10-15 17:56:40,967][52833] Updated weights for policy 0, policy_version 79100 (0.0011) -[2023-10-15 17:56:41,948][52866] Updated weights for policy 1, policy_version 79330 (0.0008) -[2023-10-15 17:56:42,311][52866] Updated weights for policy 1, policy_version 79340 (0.0007) -[2023-10-15 17:56:42,676][52866] Updated weights for policy 1, policy_version 79350 (0.0008) -[2023-10-15 17:56:43,038][52866] Updated weights for policy 1, policy_version 79360 (0.0007) -[2023-10-15 17:56:43,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 162267136. Throughput: 0: 1808.6, 1: 1810.8. Samples: 40574738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:56:43,442][51532] Avg episode reward: [(0, '70.560'), (1, '65.150')] -[2023-10-15 17:56:44,839][52833] Updated weights for policy 0, policy_version 79110 (0.0010) -[2023-10-15 17:56:45,208][52833] Updated weights for policy 0, policy_version 79120 (0.0010) -[2023-10-15 17:56:45,575][52833] Updated weights for policy 0, policy_version 79130 (0.0009) -[2023-10-15 17:56:46,635][52866] Updated weights for policy 1, policy_version 79370 (0.0007) -[2023-10-15 17:56:47,006][52866] Updated weights for policy 1, policy_version 79380 (0.0009) -[2023-10-15 17:56:47,375][52866] Updated weights for policy 1, policy_version 79390 (0.0010) -[2023-10-15 17:56:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162332672. Throughput: 0: 1808.6, 1: 1807.1. Samples: 40585898. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:56:48,441][51532] Avg episode reward: [(0, '69.570'), (1, '65.880')] -[2023-10-15 17:56:49,292][52833] Updated weights for policy 0, policy_version 79140 (0.0009) -[2023-10-15 17:56:49,657][52833] Updated weights for policy 0, policy_version 79150 (0.0008) -[2023-10-15 17:56:50,026][52833] Updated weights for policy 0, policy_version 79160 (0.0008) -[2023-10-15 17:56:51,091][52866] Updated weights for policy 1, policy_version 79400 (0.0009) -[2023-10-15 17:56:51,462][52866] Updated weights for policy 1, policy_version 79410 (0.0009) -[2023-10-15 17:56:51,827][52866] Updated weights for policy 1, policy_version 79420 (0.0008) -[2023-10-15 17:56:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162398208. Throughput: 0: 1807.1, 1: 1807.6. Samples: 40607072. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:56:53,442][51532] Avg episode reward: [(0, '69.280'), (1, '65.140')] -[2023-10-15 17:56:53,906][52833] Updated weights for policy 0, policy_version 79170 (0.0009) -[2023-10-15 17:56:54,300][52833] Updated weights for policy 0, policy_version 79180 (0.0010) -[2023-10-15 17:56:54,670][52833] Updated weights for policy 0, policy_version 79190 (0.0007) -[2023-10-15 17:56:55,031][52833] Updated weights for policy 0, policy_version 79200 (0.0008) -[2023-10-15 17:56:55,618][52866] Updated weights for policy 1, policy_version 79430 (0.0009) -[2023-10-15 17:56:55,987][52866] Updated weights for policy 1, policy_version 79440 (0.0009) -[2023-10-15 17:56:56,351][52866] Updated weights for policy 1, policy_version 79450 (0.0008) -[2023-10-15 17:56:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162463744. Throughput: 0: 1794.3, 1: 1801.7. Samples: 40628954. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:56:58,441][51532] Avg episode reward: [(0, '69.570'), (1, '63.240')] -[2023-10-15 17:56:58,762][52833] Updated weights for policy 0, policy_version 79210 (0.0008) -[2023-10-15 17:56:59,134][52833] Updated weights for policy 0, policy_version 79220 (0.0009) -[2023-10-15 17:56:59,504][52833] Updated weights for policy 0, policy_version 79230 (0.0009) -[2023-10-15 17:57:00,132][52866] Updated weights for policy 1, policy_version 79460 (0.0009) -[2023-10-15 17:57:00,521][52866] Updated weights for policy 1, policy_version 79470 (0.0007) -[2023-10-15 17:57:00,883][52866] Updated weights for policy 1, policy_version 79480 (0.0007) -[2023-10-15 17:57:03,290][52833] Updated weights for policy 0, policy_version 79240 (0.0008) -[2023-10-15 17:57:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162529280. Throughput: 0: 1794.4, 1: 1809.4. Samples: 40639086. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:57:03,442][51532] Avg episode reward: [(0, '68.840'), (1, '63.260')] -[2023-10-15 17:57:03,649][52833] Updated weights for policy 0, policy_version 79250 (0.0008) -[2023-10-15 17:57:04,025][52833] Updated weights for policy 0, policy_version 79260 (0.0008) -[2023-10-15 17:57:04,567][52866] Updated weights for policy 1, policy_version 79490 (0.0008) -[2023-10-15 17:57:04,943][52866] Updated weights for policy 1, policy_version 79500 (0.0010) -[2023-10-15 17:57:05,314][52866] Updated weights for policy 1, policy_version 79510 (0.0012) -[2023-10-15 17:57:05,681][52866] Updated weights for policy 1, policy_version 79520 (0.0011) -[2023-10-15 17:57:07,778][52833] Updated weights for policy 0, policy_version 79270 (0.0008) -[2023-10-15 17:57:08,144][52833] Updated weights for policy 0, policy_version 79280 (0.0010) -[2023-10-15 17:57:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 162594816. Throughput: 0: 1797.7, 1: 1800.3. Samples: 40661330. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:57:08,441][51532] Avg episode reward: [(0, '70.300'), (1, '63.330')] -[2023-10-15 17:57:08,524][52833] Updated weights for policy 0, policy_version 79290 (0.0008) -[2023-10-15 17:57:09,412][52866] Updated weights for policy 1, policy_version 79530 (0.0009) -[2023-10-15 17:57:09,775][52866] Updated weights for policy 1, policy_version 79540 (0.0008) -[2023-10-15 17:57:10,141][52866] Updated weights for policy 1, policy_version 79550 (0.0008) -[2023-10-15 17:57:12,215][52833] Updated weights for policy 0, policy_version 79300 (0.0008) -[2023-10-15 17:57:12,584][52833] Updated weights for policy 0, policy_version 79310 (0.0007) -[2023-10-15 17:57:12,960][52833] Updated weights for policy 0, policy_version 79320 (0.0010) -[2023-10-15 17:57:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 162693120. Throughput: 0: 1804.2, 1: 1799.8. Samples: 40682922. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:57:13,442][51532] Avg episode reward: [(0, '70.160'), (1, '65.330')] -[2023-10-15 17:57:13,906][52866] Updated weights for policy 1, policy_version 79560 (0.0009) -[2023-10-15 17:57:14,283][52866] Updated weights for policy 1, policy_version 79570 (0.0008) -[2023-10-15 17:57:14,645][52866] Updated weights for policy 1, policy_version 79580 (0.0009) -[2023-10-15 17:57:16,682][52833] Updated weights for policy 0, policy_version 79330 (0.0008) -[2023-10-15 17:57:17,043][52833] Updated weights for policy 0, policy_version 79340 (0.0007) -[2023-10-15 17:57:17,418][52833] Updated weights for policy 0, policy_version 79350 (0.0009) -[2023-10-15 17:57:17,793][52833] Updated weights for policy 0, policy_version 79360 (0.0010) -[2023-10-15 17:57:18,428][52866] Updated weights for policy 1, policy_version 79590 (0.0008) -[2023-10-15 17:57:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 162758656. Throughput: 0: 1795.9, 1: 1797.3. Samples: 40693714. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:57:18,441][51532] Avg episode reward: [(0, '71.380'), (1, '63.910')] -[2023-10-15 17:57:18,794][52866] Updated weights for policy 1, policy_version 79600 (0.0008) -[2023-10-15 17:57:19,165][52866] Updated weights for policy 1, policy_version 79610 (0.0008) -[2023-10-15 17:57:21,608][52833] Updated weights for policy 0, policy_version 79370 (0.0011) -[2023-10-15 17:57:21,981][52833] Updated weights for policy 0, policy_version 79380 (0.0010) -[2023-10-15 17:57:22,364][52833] Updated weights for policy 0, policy_version 79390 (0.0010) -[2023-10-15 17:57:22,936][52866] Updated weights for policy 1, policy_version 79620 (0.0007) -[2023-10-15 17:57:23,301][52866] Updated weights for policy 1, policy_version 79630 (0.0007) -[2023-10-15 17:57:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 162824192. Throughput: 0: 1803.9, 1: 1800.7. Samples: 40715206. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:57:23,441][51532] Avg episode reward: [(0, '68.410'), (1, '65.200')] -[2023-10-15 17:57:23,666][52866] Updated weights for policy 1, policy_version 79640 (0.0007) -[2023-10-15 17:57:26,144][52833] Updated weights for policy 0, policy_version 79400 (0.0010) -[2023-10-15 17:57:26,516][52833] Updated weights for policy 0, policy_version 79410 (0.0011) -[2023-10-15 17:57:26,881][52833] Updated weights for policy 0, policy_version 79420 (0.0010) -[2023-10-15 17:57:27,394][52866] Updated weights for policy 1, policy_version 79650 (0.0008) -[2023-10-15 17:57:27,761][52866] Updated weights for policy 1, policy_version 79660 (0.0011) -[2023-10-15 17:57:28,119][52866] Updated weights for policy 1, policy_version 79670 (0.0007) -[2023-10-15 17:57:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 162889728. Throughput: 0: 1777.3, 1: 1809.9. Samples: 40736160. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:57:28,442][51532] Avg episode reward: [(0, '67.680'), (1, '65.410')] -[2023-10-15 17:57:28,483][52866] Updated weights for policy 1, policy_version 79680 (0.0008) -[2023-10-15 17:57:30,582][52833] Updated weights for policy 0, policy_version 79430 (0.0010) -[2023-10-15 17:57:30,954][52833] Updated weights for policy 0, policy_version 79440 (0.0010) -[2023-10-15 17:57:31,315][52833] Updated weights for policy 0, policy_version 79450 (0.0009) -[2023-10-15 17:57:31,921][52866] Updated weights for policy 1, policy_version 79690 (0.0008) -[2023-10-15 17:57:32,292][52866] Updated weights for policy 1, policy_version 79700 (0.0009) -[2023-10-15 17:57:32,662][52866] Updated weights for policy 1, policy_version 79710 (0.0010) -[2023-10-15 17:57:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 162988032. Throughput: 0: 1801.0, 1: 1803.7. Samples: 40748112. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 17:57:33,441][51532] Avg episode reward: [(0, '67.850'), (1, '67.450')] -[2023-10-15 17:57:35,003][52833] Updated weights for policy 0, policy_version 79460 (0.0008) -[2023-10-15 17:57:35,384][52833] Updated weights for policy 0, policy_version 79470 (0.0008) -[2023-10-15 17:57:35,752][52833] Updated weights for policy 0, policy_version 79480 (0.0010) -[2023-10-15 17:57:36,365][52866] Updated weights for policy 1, policy_version 79720 (0.0007) -[2023-10-15 17:57:36,740][52866] Updated weights for policy 1, policy_version 79730 (0.0008) -[2023-10-15 17:57:37,112][52866] Updated weights for policy 1, policy_version 79740 (0.0009) -[2023-10-15 17:57:38,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 163053568. Throughput: 0: 1782.3, 1: 1812.8. Samples: 40768854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:57:38,442][51532] Avg episode reward: [(0, '63.930'), (1, '66.930')] -[2023-10-15 17:57:39,497][52833] Updated weights for policy 0, policy_version 79490 (0.0009) -[2023-10-15 17:57:39,908][52833] Updated weights for policy 0, policy_version 79500 (0.0010) -[2023-10-15 17:57:40,283][52833] Updated weights for policy 0, policy_version 79510 (0.0011) -[2023-10-15 17:57:40,652][52833] Updated weights for policy 0, policy_version 79520 (0.0009) -[2023-10-15 17:57:40,804][52866] Updated weights for policy 1, policy_version 79750 (0.0008) -[2023-10-15 17:57:41,164][52866] Updated weights for policy 1, policy_version 79760 (0.0008) -[2023-10-15 17:57:41,525][52866] Updated weights for policy 1, policy_version 79770 (0.0008) -[2023-10-15 17:57:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163119104. Throughput: 0: 1789.1, 1: 1806.9. Samples: 40790774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:57:43,441][51532] Avg episode reward: [(0, '65.880'), (1, '69.830')] -[2023-10-15 17:57:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000079776_81690624.pth... -[2023-10-15 17:57:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000079520_81428480.pth... -[2023-10-15 17:57:43,488][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000077856_79724544.pth -[2023-10-15 17:57:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000078080_79953920.pth -[2023-10-15 17:57:44,484][52833] Updated weights for policy 0, policy_version 79530 (0.0011) -[2023-10-15 17:57:44,854][52833] Updated weights for policy 0, policy_version 79540 (0.0010) -[2023-10-15 17:57:45,219][52833] Updated weights for policy 0, policy_version 79550 (0.0008) -[2023-10-15 17:57:45,348][52866] Updated weights for policy 1, policy_version 79780 (0.0009) -[2023-10-15 17:57:45,743][52866] Updated weights for policy 1, policy_version 79790 (0.0010) -[2023-10-15 17:57:46,113][52866] Updated weights for policy 1, policy_version 79800 (0.0007) -[2023-10-15 17:57:48,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163184640. Throughput: 0: 1784.0, 1: 1810.6. Samples: 40800842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:57:48,441][51532] Avg episode reward: [(0, '65.210'), (1, '69.670')] -[2023-10-15 17:57:49,081][52833] Updated weights for policy 0, policy_version 79560 (0.0010) -[2023-10-15 17:57:49,450][52833] Updated weights for policy 0, policy_version 79570 (0.0011) -[2023-10-15 17:57:49,820][52833] Updated weights for policy 0, policy_version 79580 (0.0008) -[2023-10-15 17:57:49,860][52866] Updated weights for policy 1, policy_version 79810 (0.0010) -[2023-10-15 17:57:50,220][52866] Updated weights for policy 1, policy_version 79820 (0.0008) -[2023-10-15 17:57:50,588][52866] Updated weights for policy 1, policy_version 79830 (0.0008) -[2023-10-15 17:57:50,951][52866] Updated weights for policy 1, policy_version 79840 (0.0010) -[2023-10-15 17:57:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163250176. Throughput: 0: 1785.1, 1: 1801.9. Samples: 40822742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:57:53,441][51532] Avg episode reward: [(0, '65.830'), (1, '68.690')] -[2023-10-15 17:57:53,543][52833] Updated weights for policy 0, policy_version 79590 (0.0008) -[2023-10-15 17:57:53,908][52833] Updated weights for policy 0, policy_version 79600 (0.0009) -[2023-10-15 17:57:54,285][52833] Updated weights for policy 0, policy_version 79610 (0.0010) -[2023-10-15 17:57:54,751][52866] Updated weights for policy 1, policy_version 79850 (0.0009) -[2023-10-15 17:57:55,126][52866] Updated weights for policy 1, policy_version 79860 (0.0009) -[2023-10-15 17:57:55,495][52866] Updated weights for policy 1, policy_version 79870 (0.0009) -[2023-10-15 17:57:58,035][52833] Updated weights for policy 0, policy_version 79620 (0.0009) -[2023-10-15 17:57:58,409][52833] Updated weights for policy 0, policy_version 79630 (0.0008) -[2023-10-15 17:57:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 163315712. Throughput: 0: 1814.4, 1: 1795.3. Samples: 40845356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:57:58,441][51532] Avg episode reward: [(0, '63.970'), (1, '69.780')] -[2023-10-15 17:57:58,780][52833] Updated weights for policy 0, policy_version 79640 (0.0008) -[2023-10-15 17:57:59,351][52866] Updated weights for policy 1, policy_version 79880 (0.0010) -[2023-10-15 17:57:59,722][52866] Updated weights for policy 1, policy_version 79890 (0.0009) -[2023-10-15 17:58:00,089][52866] Updated weights for policy 1, policy_version 79900 (0.0008) -[2023-10-15 17:58:02,481][52833] Updated weights for policy 0, policy_version 79650 (0.0009) -[2023-10-15 17:58:02,853][52833] Updated weights for policy 0, policy_version 79660 (0.0008) -[2023-10-15 17:58:03,216][52833] Updated weights for policy 0, policy_version 79670 (0.0007) -[2023-10-15 17:58:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 163381248. Throughput: 0: 1790.7, 1: 1801.7. Samples: 40855372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:58:03,442][51532] Avg episode reward: [(0, '66.810'), (1, '72.980')] -[2023-10-15 17:58:03,585][52833] Updated weights for policy 0, policy_version 79680 (0.0007) -[2023-10-15 17:58:03,739][52866] Updated weights for policy 1, policy_version 79910 (0.0009) -[2023-10-15 17:58:04,098][52866] Updated weights for policy 1, policy_version 79920 (0.0011) -[2023-10-15 17:58:04,462][52866] Updated weights for policy 1, policy_version 79930 (0.0009) -[2023-10-15 17:58:07,117][52833] Updated weights for policy 0, policy_version 79690 (0.0010) -[2023-10-15 17:58:07,480][52833] Updated weights for policy 0, policy_version 79700 (0.0009) -[2023-10-15 17:58:07,852][52833] Updated weights for policy 0, policy_version 79710 (0.0007) -[2023-10-15 17:58:08,131][52866] Updated weights for policy 1, policy_version 79940 (0.0011) -[2023-10-15 17:58:08,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 163479552. Throughput: 0: 1812.0, 1: 1807.9. Samples: 40878102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:58:08,441][51532] Avg episode reward: [(0, '64.580'), (1, '71.850')] -[2023-10-15 17:58:08,491][52866] Updated weights for policy 1, policy_version 79950 (0.0008) -[2023-10-15 17:58:08,854][52866] Updated weights for policy 1, policy_version 79960 (0.0007) -[2023-10-15 17:58:11,639][52833] Updated weights for policy 0, policy_version 79720 (0.0010) -[2023-10-15 17:58:12,011][52833] Updated weights for policy 0, policy_version 79730 (0.0009) -[2023-10-15 17:58:12,390][52833] Updated weights for policy 0, policy_version 79740 (0.0009) -[2023-10-15 17:58:12,667][52866] Updated weights for policy 1, policy_version 79970 (0.0008) -[2023-10-15 17:58:13,034][52866] Updated weights for policy 1, policy_version 79980 (0.0007) -[2023-10-15 17:58:13,406][52866] Updated weights for policy 1, policy_version 79990 (0.0008) -[2023-10-15 17:58:13,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163545088. Throughput: 0: 1802.8, 1: 1811.8. Samples: 40898814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:58:13,441][51532] Avg episode reward: [(0, '63.480'), (1, '72.870')] -[2023-10-15 17:58:13,768][52866] Updated weights for policy 1, policy_version 80000 (0.0009) -[2023-10-15 17:58:16,149][52833] Updated weights for policy 0, policy_version 79750 (0.0009) -[2023-10-15 17:58:16,522][52833] Updated weights for policy 0, policy_version 79760 (0.0008) -[2023-10-15 17:58:16,886][52833] Updated weights for policy 0, policy_version 79770 (0.0007) -[2023-10-15 17:58:17,527][52866] Updated weights for policy 1, policy_version 80010 (0.0008) -[2023-10-15 17:58:17,904][52866] Updated weights for policy 1, policy_version 80020 (0.0009) -[2023-10-15 17:58:18,268][52866] Updated weights for policy 1, policy_version 80030 (0.0008) -[2023-10-15 17:58:18,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 163643392. Throughput: 0: 1811.2, 1: 1794.4. Samples: 40910368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:58:18,441][51532] Avg episode reward: [(0, '63.140'), (1, '70.440')] -[2023-10-15 17:58:20,620][52833] Updated weights for policy 0, policy_version 79780 (0.0008) -[2023-10-15 17:58:20,988][52833] Updated weights for policy 0, policy_version 79790 (0.0009) -[2023-10-15 17:58:21,352][52833] Updated weights for policy 0, policy_version 79800 (0.0008) -[2023-10-15 17:58:21,925][52866] Updated weights for policy 1, policy_version 80040 (0.0008) -[2023-10-15 17:58:22,295][52866] Updated weights for policy 1, policy_version 80050 (0.0007) -[2023-10-15 17:58:22,666][52866] Updated weights for policy 1, policy_version 80060 (0.0008) -[2023-10-15 17:58:23,441][51532] Fps is (10 sec: 16383.0, 60 sec: 14745.4, 300 sec: 14440.1). Total num frames: 163708928. Throughput: 0: 1796.4, 1: 1808.4. Samples: 40931072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 17:58:23,443][51532] Avg episode reward: [(0, '63.210'), (1, '66.460')] -[2023-10-15 17:58:25,123][52833] Updated weights for policy 0, policy_version 79810 (0.0008) -[2023-10-15 17:58:25,524][52833] Updated weights for policy 0, policy_version 79820 (0.0009) -[2023-10-15 17:58:25,898][52833] Updated weights for policy 0, policy_version 79830 (0.0008) -[2023-10-15 17:58:26,268][52833] Updated weights for policy 0, policy_version 79840 (0.0009) -[2023-10-15 17:58:26,410][52866] Updated weights for policy 1, policy_version 80070 (0.0010) -[2023-10-15 17:58:26,761][52866] Updated weights for policy 1, policy_version 80080 (0.0008) -[2023-10-15 17:58:27,128][52866] Updated weights for policy 1, policy_version 80090 (0.0007) -[2023-10-15 17:58:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 163774464. Throughput: 0: 1800.7, 1: 1794.6. Samples: 40952562. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:58:28,442][51532] Avg episode reward: [(0, '63.750'), (1, '70.030')] -[2023-10-15 17:58:29,901][52833] Updated weights for policy 0, policy_version 79850 (0.0008) -[2023-10-15 17:58:30,280][52833] Updated weights for policy 0, policy_version 79860 (0.0009) -[2023-10-15 17:58:30,649][52833] Updated weights for policy 0, policy_version 79870 (0.0009) -[2023-10-15 17:58:30,792][52866] Updated weights for policy 1, policy_version 80100 (0.0008) -[2023-10-15 17:58:31,153][52866] Updated weights for policy 1, policy_version 80110 (0.0007) -[2023-10-15 17:58:31,512][52866] Updated weights for policy 1, policy_version 80120 (0.0008) -[2023-10-15 17:58:33,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 163840000. Throughput: 0: 1800.2, 1: 1809.7. Samples: 40963286. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:58:33,442][51532] Avg episode reward: [(0, '59.700'), (1, '64.260')] -[2023-10-15 17:58:34,417][52833] Updated weights for policy 0, policy_version 79880 (0.0008) -[2023-10-15 17:58:34,784][52833] Updated weights for policy 0, policy_version 79890 (0.0010) -[2023-10-15 17:58:35,154][52833] Updated weights for policy 0, policy_version 79900 (0.0008) -[2023-10-15 17:58:35,334][52866] Updated weights for policy 1, policy_version 80130 (0.0008) -[2023-10-15 17:58:35,708][52866] Updated weights for policy 1, policy_version 80140 (0.0008) -[2023-10-15 17:58:36,067][52866] Updated weights for policy 1, policy_version 80150 (0.0007) -[2023-10-15 17:58:36,428][52866] Updated weights for policy 1, policy_version 80160 (0.0007) -[2023-10-15 17:58:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163905536. Throughput: 0: 1799.9, 1: 1798.6. Samples: 40984678. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:58:38,442][51532] Avg episode reward: [(0, '60.560'), (1, '61.080')] -[2023-10-15 17:58:38,871][52833] Updated weights for policy 0, policy_version 79910 (0.0007) -[2023-10-15 17:58:39,236][52833] Updated weights for policy 0, policy_version 79920 (0.0009) -[2023-10-15 17:58:39,605][52833] Updated weights for policy 0, policy_version 79930 (0.0009) -[2023-10-15 17:58:40,155][52866] Updated weights for policy 1, policy_version 80170 (0.0008) -[2023-10-15 17:58:40,532][52866] Updated weights for policy 1, policy_version 80180 (0.0009) -[2023-10-15 17:58:40,899][52866] Updated weights for policy 1, policy_version 80190 (0.0008) -[2023-10-15 17:58:43,295][52833] Updated weights for policy 0, policy_version 79940 (0.0009) -[2023-10-15 17:58:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163971072. Throughput: 0: 1790.1, 1: 1804.4. Samples: 41007112. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:58:43,441][51532] Avg episode reward: [(0, '56.950'), (1, '58.700')] -[2023-10-15 17:58:43,662][52833] Updated weights for policy 0, policy_version 79950 (0.0008) -[2023-10-15 17:58:44,034][52833] Updated weights for policy 0, policy_version 79960 (0.0008) -[2023-10-15 17:58:44,700][52866] Updated weights for policy 1, policy_version 80200 (0.0008) -[2023-10-15 17:58:45,059][52866] Updated weights for policy 1, policy_version 80210 (0.0008) -[2023-10-15 17:58:45,424][52866] Updated weights for policy 1, policy_version 80220 (0.0007) -[2023-10-15 17:58:47,866][52833] Updated weights for policy 0, policy_version 79970 (0.0009) -[2023-10-15 17:58:48,234][52833] Updated weights for policy 0, policy_version 79980 (0.0007) -[2023-10-15 17:58:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 164036608. Throughput: 0: 1789.7, 1: 1801.3. Samples: 41016966. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:58:48,441][51532] Avg episode reward: [(0, '53.420'), (1, '59.730')] -[2023-10-15 17:58:48,607][52833] Updated weights for policy 0, policy_version 79990 (0.0009) -[2023-10-15 17:58:48,973][52833] Updated weights for policy 0, policy_version 80000 (0.0007) -[2023-10-15 17:58:49,104][52866] Updated weights for policy 1, policy_version 80230 (0.0008) -[2023-10-15 17:58:49,468][52866] Updated weights for policy 1, policy_version 80240 (0.0007) -[2023-10-15 17:58:49,833][52866] Updated weights for policy 1, policy_version 80250 (0.0008) -[2023-10-15 17:58:52,886][52833] Updated weights for policy 0, policy_version 80010 (0.0009) -[2023-10-15 17:58:53,254][52833] Updated weights for policy 0, policy_version 80020 (0.0008) -[2023-10-15 17:58:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 164102144. Throughput: 0: 1790.3, 1: 1795.6. Samples: 41039466. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:58:53,441][51532] Avg episode reward: [(0, '51.970'), (1, '59.730')] -[2023-10-15 17:58:53,493][52866] Updated weights for policy 1, policy_version 80260 (0.0009) -[2023-10-15 17:58:53,632][52833] Updated weights for policy 0, policy_version 80030 (0.0009) -[2023-10-15 17:58:53,859][52866] Updated weights for policy 1, policy_version 80270 (0.0010) -[2023-10-15 17:58:54,234][52866] Updated weights for policy 1, policy_version 80280 (0.0009) -[2023-10-15 17:58:57,493][52833] Updated weights for policy 0, policy_version 80040 (0.0008) -[2023-10-15 17:58:57,872][52833] Updated weights for policy 0, policy_version 80050 (0.0008) -[2023-10-15 17:58:57,989][52866] Updated weights for policy 1, policy_version 80290 (0.0009) -[2023-10-15 17:58:58,233][52833] Updated weights for policy 0, policy_version 80060 (0.0008) -[2023-10-15 17:58:58,357][52866] Updated weights for policy 1, policy_version 80300 (0.0008) -[2023-10-15 17:58:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 164200448. Throughput: 0: 1797.2, 1: 1810.0. Samples: 41061136. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:58:58,441][51532] Avg episode reward: [(0, '54.840'), (1, '59.000')] -[2023-10-15 17:58:58,713][52866] Updated weights for policy 1, policy_version 80310 (0.0010) -[2023-10-15 17:58:59,075][52866] Updated weights for policy 1, policy_version 80320 (0.0008) -[2023-10-15 17:59:01,913][52833] Updated weights for policy 0, policy_version 80070 (0.0009) -[2023-10-15 17:59:02,289][52833] Updated weights for policy 0, policy_version 80080 (0.0010) -[2023-10-15 17:59:02,653][52833] Updated weights for policy 0, policy_version 80090 (0.0008) -[2023-10-15 17:59:02,863][52866] Updated weights for policy 1, policy_version 80330 (0.0010) -[2023-10-15 17:59:03,228][52866] Updated weights for policy 1, policy_version 80340 (0.0011) -[2023-10-15 17:59:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 164265984. Throughput: 0: 1785.3, 1: 1799.1. Samples: 41071666. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:59:03,441][51532] Avg episode reward: [(0, '54.790'), (1, '60.020')] -[2023-10-15 17:59:03,594][52866] Updated weights for policy 1, policy_version 80350 (0.0007) -[2023-10-15 17:59:06,581][52833] Updated weights for policy 0, policy_version 80100 (0.0008) -[2023-10-15 17:59:06,956][52833] Updated weights for policy 0, policy_version 80110 (0.0008) -[2023-10-15 17:59:07,324][52833] Updated weights for policy 0, policy_version 80120 (0.0009) -[2023-10-15 17:59:07,458][52866] Updated weights for policy 1, policy_version 80360 (0.0009) -[2023-10-15 17:59:07,819][52866] Updated weights for policy 1, policy_version 80370 (0.0009) -[2023-10-15 17:59:08,200][52866] Updated weights for policy 1, policy_version 80380 (0.0009) -[2023-10-15 17:59:08,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 164364288. Throughput: 0: 1805.3, 1: 1805.9. Samples: 41093574. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:59:08,442][51532] Avg episode reward: [(0, '57.760'), (1, '60.140')] -[2023-10-15 17:59:11,065][52833] Updated weights for policy 0, policy_version 80130 (0.0008) -[2023-10-15 17:59:11,478][52833] Updated weights for policy 0, policy_version 80140 (0.0010) -[2023-10-15 17:59:11,772][52866] Updated weights for policy 1, policy_version 80390 (0.0008) -[2023-10-15 17:59:11,846][52833] Updated weights for policy 0, policy_version 80150 (0.0009) -[2023-10-15 17:59:12,137][52866] Updated weights for policy 1, policy_version 80400 (0.0009) -[2023-10-15 17:59:12,214][52833] Updated weights for policy 0, policy_version 80160 (0.0009) -[2023-10-15 17:59:12,495][52866] Updated weights for policy 1, policy_version 80410 (0.0010) -[2023-10-15 17:59:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 164429824. Throughput: 0: 1778.2, 1: 1798.0. Samples: 41113494. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) -[2023-10-15 17:59:13,441][51532] Avg episode reward: [(0, '56.030'), (1, '59.650')] -[2023-10-15 17:59:15,894][52833] Updated weights for policy 0, policy_version 80170 (0.0010) -[2023-10-15 17:59:16,261][52833] Updated weights for policy 0, policy_version 80180 (0.0009) -[2023-10-15 17:59:16,274][52866] Updated weights for policy 1, policy_version 80420 (0.0009) -[2023-10-15 17:59:16,627][52833] Updated weights for policy 0, policy_version 80190 (0.0008) -[2023-10-15 17:59:16,652][52866] Updated weights for policy 1, policy_version 80430 (0.0008) -[2023-10-15 17:59:17,030][52866] Updated weights for policy 1, policy_version 80440 (0.0009) -[2023-10-15 17:59:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 164495360. Throughput: 0: 1809.4, 1: 1806.6. Samples: 41126006. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:18,442][51532] Avg episode reward: [(0, '57.940'), (1, '59.420')] -[2023-10-15 17:59:20,283][52833] Updated weights for policy 0, policy_version 80200 (0.0008) -[2023-10-15 17:59:20,643][52833] Updated weights for policy 0, policy_version 80210 (0.0007) -[2023-10-15 17:59:20,903][52866] Updated weights for policy 1, policy_version 80450 (0.0008) -[2023-10-15 17:59:21,022][52833] Updated weights for policy 0, policy_version 80220 (0.0008) -[2023-10-15 17:59:21,268][52866] Updated weights for policy 1, policy_version 80460 (0.0010) -[2023-10-15 17:59:21,636][52866] Updated weights for policy 1, policy_version 80470 (0.0009) -[2023-10-15 17:59:22,000][52866] Updated weights for policy 1, policy_version 80480 (0.0007) -[2023-10-15 17:59:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 164560896. Throughput: 0: 1786.0, 1: 1795.4. Samples: 41145840. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:23,442][51532] Avg episode reward: [(0, '60.120'), (1, '59.910')] -[2023-10-15 17:59:24,572][52833] Updated weights for policy 0, policy_version 80230 (0.0008) -[2023-10-15 17:59:24,945][52833] Updated weights for policy 0, policy_version 80240 (0.0010) -[2023-10-15 17:59:25,307][52833] Updated weights for policy 0, policy_version 80250 (0.0007) -[2023-10-15 17:59:25,672][52866] Updated weights for policy 1, policy_version 80490 (0.0007) -[2023-10-15 17:59:26,036][52866] Updated weights for policy 1, policy_version 80500 (0.0008) -[2023-10-15 17:59:26,405][52866] Updated weights for policy 1, policy_version 80510 (0.0010) -[2023-10-15 17:59:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 164626432. Throughput: 0: 1790.0, 1: 1789.9. Samples: 41168206. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:28,442][51532] Avg episode reward: [(0, '61.160'), (1, '60.900')] -[2023-10-15 17:59:29,127][52833] Updated weights for policy 0, policy_version 80260 (0.0009) -[2023-10-15 17:59:29,501][52833] Updated weights for policy 0, policy_version 80270 (0.0009) -[2023-10-15 17:59:29,868][52833] Updated weights for policy 0, policy_version 80280 (0.0009) -[2023-10-15 17:59:30,260][52866] Updated weights for policy 1, policy_version 80520 (0.0008) -[2023-10-15 17:59:30,630][52866] Updated weights for policy 1, policy_version 80530 (0.0008) -[2023-10-15 17:59:30,990][52866] Updated weights for policy 1, policy_version 80540 (0.0010) -[2023-10-15 17:59:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 164691968. Throughput: 0: 1789.2, 1: 1794.8. Samples: 41178250. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:33,442][51532] Avg episode reward: [(0, '63.340'), (1, '60.980')] -[2023-10-15 17:59:33,574][52833] Updated weights for policy 0, policy_version 80290 (0.0007) -[2023-10-15 17:59:33,949][52833] Updated weights for policy 0, policy_version 80300 (0.0008) -[2023-10-15 17:59:34,313][52833] Updated weights for policy 0, policy_version 80310 (0.0008) -[2023-10-15 17:59:34,684][52833] Updated weights for policy 0, policy_version 80320 (0.0008) -[2023-10-15 17:59:34,846][52866] Updated weights for policy 1, policy_version 80550 (0.0008) -[2023-10-15 17:59:35,215][52866] Updated weights for policy 1, policy_version 80560 (0.0009) -[2023-10-15 17:59:35,592][52866] Updated weights for policy 1, policy_version 80570 (0.0008) -[2023-10-15 17:59:38,409][52833] Updated weights for policy 0, policy_version 80330 (0.0007) -[2023-10-15 17:59:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 164757504. Throughput: 0: 1792.2, 1: 1786.6. Samples: 41200510. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:38,441][51532] Avg episode reward: [(0, '64.870'), (1, '63.590')] -[2023-10-15 17:59:38,777][52833] Updated weights for policy 0, policy_version 80340 (0.0007) -[2023-10-15 17:59:39,143][52833] Updated weights for policy 0, policy_version 80350 (0.0007) -[2023-10-15 17:59:39,329][52866] Updated weights for policy 1, policy_version 80580 (0.0007) -[2023-10-15 17:59:39,705][52866] Updated weights for policy 1, policy_version 80590 (0.0009) -[2023-10-15 17:59:40,066][52866] Updated weights for policy 1, policy_version 80600 (0.0009) -[2023-10-15 17:59:42,908][52833] Updated weights for policy 0, policy_version 80360 (0.0008) -[2023-10-15 17:59:43,274][52833] Updated weights for policy 0, policy_version 80370 (0.0007) -[2023-10-15 17:59:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 164823040. Throughput: 0: 1810.0, 1: 1788.8. Samples: 41223078. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:43,442][51532] Avg episode reward: [(0, '64.990'), (1, '65.520')] -[2023-10-15 17:59:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000080608_82542592.pth... -[2023-10-15 17:59:43,480][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000078944_80838656.pth -[2023-10-15 17:59:43,650][52833] Updated weights for policy 0, policy_version 80380 (0.0009) -[2023-10-15 17:59:43,743][52866] Updated weights for policy 1, policy_version 80610 (0.0008) -[2023-10-15 17:59:43,790][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000080384_82313216.pth... -[2023-10-15 17:59:43,829][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000078688_80576512.pth -[2023-10-15 17:59:44,109][52866] Updated weights for policy 1, policy_version 80620 (0.0008) -[2023-10-15 17:59:44,470][52866] Updated weights for policy 1, policy_version 80630 (0.0010) -[2023-10-15 17:59:44,839][52866] Updated weights for policy 1, policy_version 80640 (0.0011) -[2023-10-15 17:59:47,406][52833] Updated weights for policy 0, policy_version 80390 (0.0008) -[2023-10-15 17:59:47,779][52833] Updated weights for policy 0, policy_version 80400 (0.0009) -[2023-10-15 17:59:48,150][52833] Updated weights for policy 0, policy_version 80410 (0.0008) -[2023-10-15 17:59:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 164921344. Throughput: 0: 1800.0, 1: 1790.6. Samples: 41233240. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:48,441][51532] Avg episode reward: [(0, '65.160'), (1, '65.700')] -[2023-10-15 17:59:48,579][52866] Updated weights for policy 1, policy_version 80650 (0.0009) -[2023-10-15 17:59:48,932][52866] Updated weights for policy 1, policy_version 80660 (0.0008) -[2023-10-15 17:59:49,299][52866] Updated weights for policy 1, policy_version 80670 (0.0007) -[2023-10-15 17:59:51,708][52833] Updated weights for policy 0, policy_version 80420 (0.0009) -[2023-10-15 17:59:52,083][52833] Updated weights for policy 0, policy_version 80430 (0.0008) -[2023-10-15 17:59:52,448][52833] Updated weights for policy 0, policy_version 80440 (0.0008) -[2023-10-15 17:59:53,076][52866] Updated weights for policy 1, policy_version 80680 (0.0007) -[2023-10-15 17:59:53,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 164986880. Throughput: 0: 1806.1, 1: 1793.9. Samples: 41255574. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:53,441][51532] Avg episode reward: [(0, '68.770'), (1, '65.440')] -[2023-10-15 17:59:53,444][52866] Updated weights for policy 1, policy_version 80690 (0.0007) -[2023-10-15 17:59:53,816][52866] Updated weights for policy 1, policy_version 80700 (0.0009) -[2023-10-15 17:59:56,238][52833] Updated weights for policy 0, policy_version 80450 (0.0008) -[2023-10-15 17:59:56,616][52833] Updated weights for policy 0, policy_version 80460 (0.0008) -[2023-10-15 17:59:56,989][52833] Updated weights for policy 0, policy_version 80470 (0.0008) -[2023-10-15 17:59:57,347][52833] Updated weights for policy 0, policy_version 80480 (0.0009) -[2023-10-15 17:59:57,550][52866] Updated weights for policy 1, policy_version 80710 (0.0011) -[2023-10-15 17:59:57,908][52866] Updated weights for policy 1, policy_version 80720 (0.0011) -[2023-10-15 17:59:58,278][52866] Updated weights for policy 1, policy_version 80730 (0.0009) -[2023-10-15 17:59:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 165052416. Throughput: 0: 1804.9, 1: 1812.5. Samples: 41276280. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 17:59:58,442][51532] Avg episode reward: [(0, '67.490'), (1, '64.900')] -[2023-10-15 18:00:01,107][52833] Updated weights for policy 0, policy_version 80490 (0.0008) -[2023-10-15 18:00:01,475][52833] Updated weights for policy 0, policy_version 80500 (0.0009) -[2023-10-15 18:00:01,841][52833] Updated weights for policy 0, policy_version 80510 (0.0007) -[2023-10-15 18:00:02,186][52866] Updated weights for policy 1, policy_version 80740 (0.0007) -[2023-10-15 18:00:02,571][52866] Updated weights for policy 1, policy_version 80750 (0.0007) -[2023-10-15 18:00:02,930][52866] Updated weights for policy 1, policy_version 80760 (0.0008) -[2023-10-15 18:00:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 165150720. Throughput: 0: 1806.2, 1: 1794.3. Samples: 41288028. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:00:03,441][51532] Avg episode reward: [(0, '64.820'), (1, '62.990')] -[2023-10-15 18:00:05,505][52833] Updated weights for policy 0, policy_version 80520 (0.0008) -[2023-10-15 18:00:05,873][52833] Updated weights for policy 0, policy_version 80530 (0.0007) -[2023-10-15 18:00:06,239][52833] Updated weights for policy 0, policy_version 80540 (0.0010) -[2023-10-15 18:00:06,680][52866] Updated weights for policy 1, policy_version 80770 (0.0008) -[2023-10-15 18:00:07,051][52866] Updated weights for policy 1, policy_version 80780 (0.0007) -[2023-10-15 18:00:07,406][52866] Updated weights for policy 1, policy_version 80790 (0.0008) -[2023-10-15 18:00:07,775][52866] Updated weights for policy 1, policy_version 80800 (0.0010) -[2023-10-15 18:00:08,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 165216256. Throughput: 0: 1799.8, 1: 1818.9. Samples: 41308680. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:00:08,442][51532] Avg episode reward: [(0, '67.840'), (1, '64.250')] -[2023-10-15 18:00:10,082][52833] Updated weights for policy 0, policy_version 80550 (0.0007) -[2023-10-15 18:00:10,448][52833] Updated weights for policy 0, policy_version 80560 (0.0009) -[2023-10-15 18:00:10,824][52833] Updated weights for policy 0, policy_version 80570 (0.0008) -[2023-10-15 18:00:11,404][52866] Updated weights for policy 1, policy_version 80810 (0.0007) -[2023-10-15 18:00:11,775][52866] Updated weights for policy 1, policy_version 80820 (0.0007) -[2023-10-15 18:00:12,148][52866] Updated weights for policy 1, policy_version 80830 (0.0007) -[2023-10-15 18:00:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 165281792. Throughput: 0: 1800.3, 1: 1802.5. Samples: 41330332. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:13,442][51532] Avg episode reward: [(0, '67.380'), (1, '64.760')] -[2023-10-15 18:00:14,443][52833] Updated weights for policy 0, policy_version 80580 (0.0009) -[2023-10-15 18:00:14,809][52833] Updated weights for policy 0, policy_version 80590 (0.0011) -[2023-10-15 18:00:15,186][52833] Updated weights for policy 0, policy_version 80600 (0.0011) -[2023-10-15 18:00:15,949][52866] Updated weights for policy 1, policy_version 80840 (0.0008) -[2023-10-15 18:00:16,316][52866] Updated weights for policy 1, policy_version 80850 (0.0007) -[2023-10-15 18:00:16,687][52866] Updated weights for policy 1, policy_version 80860 (0.0008) -[2023-10-15 18:00:18,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165347328. Throughput: 0: 1801.4, 1: 1815.8. Samples: 41341022. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:18,441][51532] Avg episode reward: [(0, '68.720'), (1, '65.260')] -[2023-10-15 18:00:18,914][52833] Updated weights for policy 0, policy_version 80610 (0.0010) -[2023-10-15 18:00:19,285][52833] Updated weights for policy 0, policy_version 80620 (0.0007) -[2023-10-15 18:00:19,657][52833] Updated weights for policy 0, policy_version 80630 (0.0007) -[2023-10-15 18:00:20,024][52833] Updated weights for policy 0, policy_version 80640 (0.0011) -[2023-10-15 18:00:20,284][52866] Updated weights for policy 1, policy_version 80870 (0.0009) -[2023-10-15 18:00:20,661][52866] Updated weights for policy 1, policy_version 80880 (0.0007) -[2023-10-15 18:00:21,022][52866] Updated weights for policy 1, policy_version 80890 (0.0009) -[2023-10-15 18:00:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165412864. Throughput: 0: 1803.1, 1: 1804.4. Samples: 41362844. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:23,441][51532] Avg episode reward: [(0, '66.390'), (1, '64.360')] -[2023-10-15 18:00:23,697][52833] Updated weights for policy 0, policy_version 80650 (0.0007) -[2023-10-15 18:00:24,061][52833] Updated weights for policy 0, policy_version 80660 (0.0008) -[2023-10-15 18:00:24,429][52833] Updated weights for policy 0, policy_version 80670 (0.0010) -[2023-10-15 18:00:24,785][52866] Updated weights for policy 1, policy_version 80900 (0.0009) -[2023-10-15 18:00:25,152][52866] Updated weights for policy 1, policy_version 80910 (0.0009) -[2023-10-15 18:00:25,523][52866] Updated weights for policy 1, policy_version 80920 (0.0007) -[2023-10-15 18:00:28,226][52833] Updated weights for policy 0, policy_version 80680 (0.0009) -[2023-10-15 18:00:28,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 165478400. Throughput: 0: 1811.4, 1: 1798.8. Samples: 41385540. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:28,442][51532] Avg episode reward: [(0, '62.550'), (1, '63.890')] -[2023-10-15 18:00:28,596][52833] Updated weights for policy 0, policy_version 80690 (0.0011) -[2023-10-15 18:00:28,966][52833] Updated weights for policy 0, policy_version 80700 (0.0008) -[2023-10-15 18:00:29,222][52866] Updated weights for policy 1, policy_version 80930 (0.0008) -[2023-10-15 18:00:29,579][52866] Updated weights for policy 1, policy_version 80940 (0.0010) -[2023-10-15 18:00:29,940][52866] Updated weights for policy 1, policy_version 80950 (0.0010) -[2023-10-15 18:00:30,305][52866] Updated weights for policy 1, policy_version 80960 (0.0010) -[2023-10-15 18:00:32,704][52833] Updated weights for policy 0, policy_version 80710 (0.0011) -[2023-10-15 18:00:33,071][52833] Updated weights for policy 0, policy_version 80720 (0.0010) -[2023-10-15 18:00:33,441][52833] Updated weights for policy 0, policy_version 80730 (0.0010) -[2023-10-15 18:00:33,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 165543936. Throughput: 0: 1803.7, 1: 1799.0. Samples: 41395364. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:33,442][51532] Avg episode reward: [(0, '61.700'), (1, '65.290')] -[2023-10-15 18:00:33,963][52866] Updated weights for policy 1, policy_version 80970 (0.0010) -[2023-10-15 18:00:34,333][52866] Updated weights for policy 1, policy_version 80980 (0.0009) -[2023-10-15 18:00:34,694][52866] Updated weights for policy 1, policy_version 80990 (0.0010) -[2023-10-15 18:00:37,088][52833] Updated weights for policy 0, policy_version 80740 (0.0008) -[2023-10-15 18:00:37,463][52833] Updated weights for policy 0, policy_version 80750 (0.0008) -[2023-10-15 18:00:37,831][52833] Updated weights for policy 0, policy_version 80760 (0.0008) -[2023-10-15 18:00:38,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 165642240. Throughput: 0: 1804.7, 1: 1790.4. Samples: 41417350. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:38,441][51532] Avg episode reward: [(0, '61.500'), (1, '66.290')] -[2023-10-15 18:00:38,515][52866] Updated weights for policy 1, policy_version 81000 (0.0008) -[2023-10-15 18:00:38,876][52866] Updated weights for policy 1, policy_version 81010 (0.0011) -[2023-10-15 18:00:39,242][52866] Updated weights for policy 1, policy_version 81020 (0.0010) -[2023-10-15 18:00:41,579][52833] Updated weights for policy 0, policy_version 80770 (0.0009) -[2023-10-15 18:00:41,963][52833] Updated weights for policy 0, policy_version 80780 (0.0009) -[2023-10-15 18:00:42,324][52833] Updated weights for policy 0, policy_version 80790 (0.0007) -[2023-10-15 18:00:42,695][52833] Updated weights for policy 0, policy_version 80800 (0.0007) -[2023-10-15 18:00:43,110][52866] Updated weights for policy 1, policy_version 81030 (0.0011) -[2023-10-15 18:00:43,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 165707776. Throughput: 0: 1803.6, 1: 1798.6. Samples: 41438380. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:43,441][51532] Avg episode reward: [(0, '61.020'), (1, '64.840')] -[2023-10-15 18:00:43,472][52866] Updated weights for policy 1, policy_version 81040 (0.0007) -[2023-10-15 18:00:43,835][52866] Updated weights for policy 1, policy_version 81050 (0.0011) -[2023-10-15 18:00:46,378][52833] Updated weights for policy 0, policy_version 80810 (0.0010) -[2023-10-15 18:00:46,754][52833] Updated weights for policy 0, policy_version 80820 (0.0008) -[2023-10-15 18:00:47,120][52833] Updated weights for policy 0, policy_version 80830 (0.0007) -[2023-10-15 18:00:47,761][52866] Updated weights for policy 1, policy_version 81060 (0.0011) -[2023-10-15 18:00:48,151][52866] Updated weights for policy 1, policy_version 81070 (0.0008) -[2023-10-15 18:00:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 165773312. Throughput: 0: 1809.8, 1: 1788.3. Samples: 41449940. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:48,441][51532] Avg episode reward: [(0, '61.060'), (1, '66.910')] -[2023-10-15 18:00:48,511][52866] Updated weights for policy 1, policy_version 81080 (0.0008) -[2023-10-15 18:00:51,008][52833] Updated weights for policy 0, policy_version 80840 (0.0008) -[2023-10-15 18:00:51,381][52833] Updated weights for policy 0, policy_version 80850 (0.0008) -[2023-10-15 18:00:51,748][52833] Updated weights for policy 0, policy_version 80860 (0.0007) -[2023-10-15 18:00:52,295][52866] Updated weights for policy 1, policy_version 81090 (0.0007) -[2023-10-15 18:00:52,653][52866] Updated weights for policy 1, policy_version 81100 (0.0008) -[2023-10-15 18:00:53,020][52866] Updated weights for policy 1, policy_version 81110 (0.0007) -[2023-10-15 18:00:53,389][52866] Updated weights for policy 1, policy_version 81120 (0.0007) -[2023-10-15 18:00:53,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 165871616. Throughput: 0: 1807.2, 1: 1803.0. Samples: 41471138. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:53,441][51532] Avg episode reward: [(0, '60.640'), (1, '66.460')] -[2023-10-15 18:00:55,345][52833] Updated weights for policy 0, policy_version 80870 (0.0008) -[2023-10-15 18:00:55,706][52833] Updated weights for policy 0, policy_version 80880 (0.0010) -[2023-10-15 18:00:56,076][52833] Updated weights for policy 0, policy_version 80890 (0.0009) -[2023-10-15 18:00:56,936][52866] Updated weights for policy 1, policy_version 81130 (0.0009) -[2023-10-15 18:00:57,307][52866] Updated weights for policy 1, policy_version 81140 (0.0011) -[2023-10-15 18:00:57,678][52866] Updated weights for policy 1, policy_version 81150 (0.0008) -[2023-10-15 18:00:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 165937152. Throughput: 0: 1806.4, 1: 1792.5. Samples: 41492278. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) -[2023-10-15 18:00:58,442][51532] Avg episode reward: [(0, '61.230'), (1, '69.390')] -[2023-10-15 18:00:59,797][52833] Updated weights for policy 0, policy_version 80900 (0.0008) -[2023-10-15 18:01:00,158][52833] Updated weights for policy 0, policy_version 80910 (0.0007) -[2023-10-15 18:01:00,526][52833] Updated weights for policy 0, policy_version 80920 (0.0008) -[2023-10-15 18:01:01,361][52866] Updated weights for policy 1, policy_version 81160 (0.0009) -[2023-10-15 18:01:01,720][52866] Updated weights for policy 1, policy_version 81170 (0.0007) -[2023-10-15 18:01:02,091][52866] Updated weights for policy 1, policy_version 81180 (0.0009) -[2023-10-15 18:01:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 166002688. Throughput: 0: 1804.8, 1: 1807.8. Samples: 41503590. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:03,441][51532] Avg episode reward: [(0, '62.270'), (1, '71.030')] -[2023-10-15 18:01:04,351][52833] Updated weights for policy 0, policy_version 80930 (0.0010) -[2023-10-15 18:01:04,711][52833] Updated weights for policy 0, policy_version 80940 (0.0009) -[2023-10-15 18:01:05,092][52833] Updated weights for policy 0, policy_version 80950 (0.0009) -[2023-10-15 18:01:05,454][52833] Updated weights for policy 0, policy_version 80960 (0.0008) -[2023-10-15 18:01:05,838][52866] Updated weights for policy 1, policy_version 81190 (0.0008) -[2023-10-15 18:01:06,196][52866] Updated weights for policy 1, policy_version 81200 (0.0007) -[2023-10-15 18:01:06,562][52866] Updated weights for policy 1, policy_version 81210 (0.0008) -[2023-10-15 18:01:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 166068224. Throughput: 0: 1802.8, 1: 1790.9. Samples: 41524560. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:08,441][51532] Avg episode reward: [(0, '59.540'), (1, '69.930')] -[2023-10-15 18:01:09,094][52833] Updated weights for policy 0, policy_version 80970 (0.0007) -[2023-10-15 18:01:09,465][52833] Updated weights for policy 0, policy_version 80980 (0.0008) -[2023-10-15 18:01:09,828][52833] Updated weights for policy 0, policy_version 80990 (0.0008) -[2023-10-15 18:01:10,218][52866] Updated weights for policy 1, policy_version 81220 (0.0007) -[2023-10-15 18:01:10,586][52866] Updated weights for policy 1, policy_version 81230 (0.0007) -[2023-10-15 18:01:10,951][52866] Updated weights for policy 1, policy_version 81240 (0.0007) -[2023-10-15 18:01:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 166133760. Throughput: 0: 1799.3, 1: 1796.8. Samples: 41547362. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:13,442][51532] Avg episode reward: [(0, '61.630'), (1, '68.870')] -[2023-10-15 18:01:13,668][52833] Updated weights for policy 0, policy_version 81000 (0.0011) -[2023-10-15 18:01:14,041][52833] Updated weights for policy 0, policy_version 81010 (0.0008) -[2023-10-15 18:01:14,401][52833] Updated weights for policy 0, policy_version 81020 (0.0010) -[2023-10-15 18:01:14,682][52866] Updated weights for policy 1, policy_version 81250 (0.0009) -[2023-10-15 18:01:15,046][52866] Updated weights for policy 1, policy_version 81260 (0.0008) -[2023-10-15 18:01:15,416][52866] Updated weights for policy 1, policy_version 81270 (0.0007) -[2023-10-15 18:01:15,774][52866] Updated weights for policy 1, policy_version 81280 (0.0008) -[2023-10-15 18:01:18,079][52833] Updated weights for policy 0, policy_version 81030 (0.0009) -[2023-10-15 18:01:18,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 166199296. Throughput: 0: 1797.2, 1: 1799.0. Samples: 41557194. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:18,442][51532] Avg episode reward: [(0, '60.200'), (1, '68.010')] -[2023-10-15 18:01:18,448][52833] Updated weights for policy 0, policy_version 81040 (0.0008) -[2023-10-15 18:01:18,818][52833] Updated weights for policy 0, policy_version 81050 (0.0007) -[2023-10-15 18:01:19,629][52866] Updated weights for policy 1, policy_version 81290 (0.0009) -[2023-10-15 18:01:20,000][52866] Updated weights for policy 1, policy_version 81300 (0.0009) -[2023-10-15 18:01:20,358][52866] Updated weights for policy 1, policy_version 81310 (0.0007) -[2023-10-15 18:01:22,550][52833] Updated weights for policy 0, policy_version 81060 (0.0008) -[2023-10-15 18:01:22,920][52833] Updated weights for policy 0, policy_version 81070 (0.0009) -[2023-10-15 18:01:23,294][52833] Updated weights for policy 0, policy_version 81080 (0.0008) -[2023-10-15 18:01:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 166264832. Throughput: 0: 1802.2, 1: 1805.3. Samples: 41579688. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:23,441][51532] Avg episode reward: [(0, '60.750'), (1, '65.930')] -[2023-10-15 18:01:24,075][52866] Updated weights for policy 1, policy_version 81320 (0.0010) -[2023-10-15 18:01:24,447][52866] Updated weights for policy 1, policy_version 81330 (0.0011) -[2023-10-15 18:01:24,820][52866] Updated weights for policy 1, policy_version 81340 (0.0010) -[2023-10-15 18:01:27,242][52833] Updated weights for policy 0, policy_version 81090 (0.0009) -[2023-10-15 18:01:27,650][52833] Updated weights for policy 0, policy_version 81100 (0.0008) -[2023-10-15 18:01:28,026][52833] Updated weights for policy 0, policy_version 81110 (0.0009) -[2023-10-15 18:01:28,386][52833] Updated weights for policy 0, policy_version 81120 (0.0009) -[2023-10-15 18:01:28,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 166363136. Throughput: 0: 1808.1, 1: 1808.1. Samples: 41601110. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:28,441][51532] Avg episode reward: [(0, '64.300'), (1, '69.160')] -[2023-10-15 18:01:28,493][52866] Updated weights for policy 1, policy_version 81350 (0.0010) -[2023-10-15 18:01:28,860][52866] Updated weights for policy 1, policy_version 81360 (0.0010) -[2023-10-15 18:01:29,234][52866] Updated weights for policy 1, policy_version 81370 (0.0010) -[2023-10-15 18:01:32,251][52833] Updated weights for policy 0, policy_version 81130 (0.0007) -[2023-10-15 18:01:32,623][52833] Updated weights for policy 0, policy_version 81140 (0.0007) -[2023-10-15 18:01:32,917][52866] Updated weights for policy 1, policy_version 81380 (0.0008) -[2023-10-15 18:01:32,987][52833] Updated weights for policy 0, policy_version 81150 (0.0008) -[2023-10-15 18:01:33,282][52866] Updated weights for policy 1, policy_version 81390 (0.0010) -[2023-10-15 18:01:33,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 166428672. Throughput: 0: 1790.4, 1: 1807.6. Samples: 41611850. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:33,441][51532] Avg episode reward: [(0, '63.840'), (1, '69.540')] -[2023-10-15 18:01:33,647][52866] Updated weights for policy 1, policy_version 81400 (0.0007) -[2023-10-15 18:01:36,713][52833] Updated weights for policy 0, policy_version 81160 (0.0008) -[2023-10-15 18:01:37,076][52833] Updated weights for policy 0, policy_version 81170 (0.0007) -[2023-10-15 18:01:37,358][52866] Updated weights for policy 1, policy_version 81410 (0.0010) -[2023-10-15 18:01:37,448][52833] Updated weights for policy 0, policy_version 81180 (0.0009) -[2023-10-15 18:01:37,726][52866] Updated weights for policy 1, policy_version 81420 (0.0009) -[2023-10-15 18:01:38,085][52866] Updated weights for policy 1, policy_version 81430 (0.0009) -[2023-10-15 18:01:38,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 166494208. Throughput: 0: 1808.6, 1: 1807.7. Samples: 41633872. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:38,442][51532] Avg episode reward: [(0, '61.220'), (1, '66.130')] -[2023-10-15 18:01:38,447][52866] Updated weights for policy 1, policy_version 81440 (0.0010) -[2023-10-15 18:01:41,242][52833] Updated weights for policy 0, policy_version 81190 (0.0008) -[2023-10-15 18:01:41,607][52833] Updated weights for policy 0, policy_version 81200 (0.0007) -[2023-10-15 18:01:41,979][52833] Updated weights for policy 0, policy_version 81210 (0.0007) -[2023-10-15 18:01:42,282][52866] Updated weights for policy 1, policy_version 81450 (0.0009) -[2023-10-15 18:01:42,650][52866] Updated weights for policy 1, policy_version 81460 (0.0008) -[2023-10-15 18:01:43,016][52866] Updated weights for policy 1, policy_version 81470 (0.0009) -[2023-10-15 18:01:43,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 166592512. Throughput: 0: 1787.9, 1: 1808.0. Samples: 41654096. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:43,442][51532] Avg episode reward: [(0, '61.190'), (1, '69.760')] -[2023-10-15 18:01:43,453][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000081216_83165184.pth... -[2023-10-15 18:01:43,453][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000081472_83427328.pth... -[2023-10-15 18:01:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000079776_81690624.pth -[2023-10-15 18:01:43,494][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000079520_81428480.pth -[2023-10-15 18:01:45,681][52833] Updated weights for policy 0, policy_version 81220 (0.0009) -[2023-10-15 18:01:46,055][52833] Updated weights for policy 0, policy_version 81230 (0.0008) -[2023-10-15 18:01:46,420][52833] Updated weights for policy 0, policy_version 81240 (0.0008) -[2023-10-15 18:01:46,936][52866] Updated weights for policy 1, policy_version 81480 (0.0007) -[2023-10-15 18:01:47,295][52866] Updated weights for policy 1, policy_version 81490 (0.0010) -[2023-10-15 18:01:47,661][52866] Updated weights for policy 1, policy_version 81500 (0.0011) -[2023-10-15 18:01:48,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 166658048. Throughput: 0: 1812.5, 1: 1799.0. Samples: 41666108. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) -[2023-10-15 18:01:48,441][51532] Avg episode reward: [(0, '61.390'), (1, '69.040')] -[2023-10-15 18:01:50,081][52833] Updated weights for policy 0, policy_version 81250 (0.0009) -[2023-10-15 18:01:50,459][52833] Updated weights for policy 0, policy_version 81260 (0.0010) -[2023-10-15 18:01:50,819][52833] Updated weights for policy 0, policy_version 81270 (0.0011) -[2023-10-15 18:01:51,189][52833] Updated weights for policy 0, policy_version 81280 (0.0009) -[2023-10-15 18:01:51,385][52866] Updated weights for policy 1, policy_version 81510 (0.0009) -[2023-10-15 18:01:51,752][52866] Updated weights for policy 1, policy_version 81520 (0.0010) -[2023-10-15 18:01:52,116][52866] Updated weights for policy 1, policy_version 81530 (0.0007) -[2023-10-15 18:01:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 166723584. Throughput: 0: 1785.9, 1: 1814.0. Samples: 41686558. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:01:53,443][51532] Avg episode reward: [(0, '63.810'), (1, '63.700')] -[2023-10-15 18:01:54,837][52833] Updated weights for policy 0, policy_version 81290 (0.0010) -[2023-10-15 18:01:55,201][52833] Updated weights for policy 0, policy_version 81300 (0.0009) -[2023-10-15 18:01:55,571][52833] Updated weights for policy 0, policy_version 81310 (0.0010) -[2023-10-15 18:01:55,759][52866] Updated weights for policy 1, policy_version 81540 (0.0008) -[2023-10-15 18:01:56,131][52866] Updated weights for policy 1, policy_version 81550 (0.0008) -[2023-10-15 18:01:56,492][52866] Updated weights for policy 1, policy_version 81560 (0.0008) -[2023-10-15 18:01:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 166789120. Throughput: 0: 1786.1, 1: 1800.5. Samples: 41708762. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:01:58,442][51532] Avg episode reward: [(0, '66.920'), (1, '60.960')] -[2023-10-15 18:01:59,301][52833] Updated weights for policy 0, policy_version 81320 (0.0008) -[2023-10-15 18:01:59,669][52833] Updated weights for policy 0, policy_version 81330 (0.0008) -[2023-10-15 18:02:00,035][52833] Updated weights for policy 0, policy_version 81340 (0.0008) -[2023-10-15 18:02:00,167][52866] Updated weights for policy 1, policy_version 81570 (0.0008) -[2023-10-15 18:02:00,528][52866] Updated weights for policy 1, policy_version 81580 (0.0008) -[2023-10-15 18:02:00,900][52866] Updated weights for policy 1, policy_version 81590 (0.0010) -[2023-10-15 18:02:01,264][52866] Updated weights for policy 1, policy_version 81600 (0.0009) -[2023-10-15 18:02:03,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 166854656. Throughput: 0: 1786.7, 1: 1809.0. Samples: 41718998. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:03,442][51532] Avg episode reward: [(0, '68.690'), (1, '61.150')] -[2023-10-15 18:02:03,666][52833] Updated weights for policy 0, policy_version 81350 (0.0008) -[2023-10-15 18:02:04,039][52833] Updated weights for policy 0, policy_version 81360 (0.0009) -[2023-10-15 18:02:04,417][52833] Updated weights for policy 0, policy_version 81370 (0.0007) -[2023-10-15 18:02:04,868][52866] Updated weights for policy 1, policy_version 81610 (0.0008) -[2023-10-15 18:02:05,239][52866] Updated weights for policy 1, policy_version 81620 (0.0008) -[2023-10-15 18:02:05,607][52866] Updated weights for policy 1, policy_version 81630 (0.0008) -[2023-10-15 18:02:08,140][52833] Updated weights for policy 0, policy_version 81380 (0.0008) -[2023-10-15 18:02:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 166920192. Throughput: 0: 1790.6, 1: 1799.9. Samples: 41741260. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:08,442][51532] Avg episode reward: [(0, '65.820'), (1, '62.770')] -[2023-10-15 18:02:08,507][52833] Updated weights for policy 0, policy_version 81390 (0.0007) -[2023-10-15 18:02:08,873][52833] Updated weights for policy 0, policy_version 81400 (0.0009) -[2023-10-15 18:02:09,547][52866] Updated weights for policy 1, policy_version 81640 (0.0007) -[2023-10-15 18:02:09,913][52866] Updated weights for policy 1, policy_version 81650 (0.0007) -[2023-10-15 18:02:10,277][52866] Updated weights for policy 1, policy_version 81660 (0.0007) -[2023-10-15 18:02:12,707][52833] Updated weights for policy 0, policy_version 81410 (0.0008) -[2023-10-15 18:02:13,099][52833] Updated weights for policy 0, policy_version 81420 (0.0009) -[2023-10-15 18:02:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 166985728. Throughput: 0: 1804.4, 1: 1798.6. Samples: 41763244. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:13,442][51532] Avg episode reward: [(0, '67.600'), (1, '66.810')] -[2023-10-15 18:02:13,463][52833] Updated weights for policy 0, policy_version 81430 (0.0008) -[2023-10-15 18:02:13,826][52833] Updated weights for policy 0, policy_version 81440 (0.0009) -[2023-10-15 18:02:13,991][52866] Updated weights for policy 1, policy_version 81670 (0.0008) -[2023-10-15 18:02:14,354][52866] Updated weights for policy 1, policy_version 81680 (0.0008) -[2023-10-15 18:02:14,720][52866] Updated weights for policy 1, policy_version 81690 (0.0011) -[2023-10-15 18:02:17,657][52833] Updated weights for policy 0, policy_version 81450 (0.0008) -[2023-10-15 18:02:18,032][52833] Updated weights for policy 0, policy_version 81460 (0.0007) -[2023-10-15 18:02:18,396][52833] Updated weights for policy 0, policy_version 81470 (0.0008) -[2023-10-15 18:02:18,428][52866] Updated weights for policy 1, policy_version 81700 (0.0010) -[2023-10-15 18:02:18,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 167051264. Throughput: 0: 1791.1, 1: 1796.0. Samples: 41773268. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:18,441][51532] Avg episode reward: [(0, '69.380'), (1, '66.810')] -[2023-10-15 18:02:18,790][52866] Updated weights for policy 1, policy_version 81710 (0.0007) -[2023-10-15 18:02:19,158][52866] Updated weights for policy 1, policy_version 81720 (0.0010) -[2023-10-15 18:02:21,985][52833] Updated weights for policy 0, policy_version 81480 (0.0009) -[2023-10-15 18:02:22,349][52833] Updated weights for policy 0, policy_version 81490 (0.0007) -[2023-10-15 18:02:22,712][52833] Updated weights for policy 0, policy_version 81500 (0.0008) -[2023-10-15 18:02:23,048][52866] Updated weights for policy 1, policy_version 81730 (0.0009) -[2023-10-15 18:02:23,421][52866] Updated weights for policy 1, policy_version 81740 (0.0009) -[2023-10-15 18:02:23,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 167149568. Throughput: 0: 1801.5, 1: 1787.8. Samples: 41795390. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:23,442][51532] Avg episode reward: [(0, '69.190'), (1, '66.300')] -[2023-10-15 18:02:23,786][52866] Updated weights for policy 1, policy_version 81750 (0.0008) -[2023-10-15 18:02:24,148][52866] Updated weights for policy 1, policy_version 81760 (0.0008) -[2023-10-15 18:02:26,552][52833] Updated weights for policy 0, policy_version 81510 (0.0008) -[2023-10-15 18:02:26,918][52833] Updated weights for policy 0, policy_version 81520 (0.0007) -[2023-10-15 18:02:27,286][52833] Updated weights for policy 0, policy_version 81530 (0.0007) -[2023-10-15 18:02:27,804][52866] Updated weights for policy 1, policy_version 81770 (0.0008) -[2023-10-15 18:02:28,167][52866] Updated weights for policy 1, policy_version 81780 (0.0007) -[2023-10-15 18:02:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 167215104. Throughput: 0: 1796.9, 1: 1807.3. Samples: 41816284. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:28,441][51532] Avg episode reward: [(0, '70.530'), (1, '66.370')] -[2023-10-15 18:02:28,535][52866] Updated weights for policy 1, policy_version 81790 (0.0008) -[2023-10-15 18:02:30,960][52833] Updated weights for policy 0, policy_version 81540 (0.0008) -[2023-10-15 18:02:31,325][52833] Updated weights for policy 0, policy_version 81550 (0.0008) -[2023-10-15 18:02:31,693][52833] Updated weights for policy 0, policy_version 81560 (0.0007) -[2023-10-15 18:02:32,395][52866] Updated weights for policy 1, policy_version 81800 (0.0008) -[2023-10-15 18:02:32,768][52866] Updated weights for policy 1, policy_version 81810 (0.0008) -[2023-10-15 18:02:33,141][52866] Updated weights for policy 1, policy_version 81820 (0.0009) -[2023-10-15 18:02:33,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 167313408. Throughput: 0: 1811.1, 1: 1794.3. Samples: 41828350. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:33,441][51532] Avg episode reward: [(0, '71.400'), (1, '66.170')] -[2023-10-15 18:02:35,434][52833] Updated weights for policy 0, policy_version 81570 (0.0007) -[2023-10-15 18:02:35,815][52833] Updated weights for policy 0, policy_version 81580 (0.0007) -[2023-10-15 18:02:36,176][52833] Updated weights for policy 0, policy_version 81590 (0.0010) -[2023-10-15 18:02:36,550][52833] Updated weights for policy 0, policy_version 81600 (0.0010) -[2023-10-15 18:02:36,911][52866] Updated weights for policy 1, policy_version 81830 (0.0009) -[2023-10-15 18:02:37,273][52866] Updated weights for policy 1, policy_version 81840 (0.0011) -[2023-10-15 18:02:37,637][52866] Updated weights for policy 1, policy_version 81850 (0.0010) -[2023-10-15 18:02:38,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 167378944. Throughput: 0: 1806.8, 1: 1807.3. Samples: 41849190. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:38,442][51532] Avg episode reward: [(0, '72.340'), (1, '64.730')] -[2023-10-15 18:02:40,355][52833] Updated weights for policy 0, policy_version 81610 (0.0010) -[2023-10-15 18:02:40,729][52833] Updated weights for policy 0, policy_version 81620 (0.0008) -[2023-10-15 18:02:41,106][52833] Updated weights for policy 0, policy_version 81630 (0.0007) -[2023-10-15 18:02:41,203][52866] Updated weights for policy 1, policy_version 81860 (0.0009) -[2023-10-15 18:02:41,578][52866] Updated weights for policy 1, policy_version 81870 (0.0010) -[2023-10-15 18:02:41,945][52866] Updated weights for policy 1, policy_version 81880 (0.0009) -[2023-10-15 18:02:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167444480. Throughput: 0: 1799.8, 1: 1794.4. Samples: 41870500. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-15 18:02:43,442][51532] Avg episode reward: [(0, '74.800'), (1, '63.070')] -[2023-10-15 18:02:44,984][52833] Updated weights for policy 0, policy_version 81640 (0.0007) -[2023-10-15 18:02:45,349][52833] Updated weights for policy 0, policy_version 81650 (0.0007) -[2023-10-15 18:02:45,723][52833] Updated weights for policy 0, policy_version 81660 (0.0009) -[2023-10-15 18:02:45,841][52866] Updated weights for policy 1, policy_version 81890 (0.0007) -[2023-10-15 18:02:46,206][52866] Updated weights for policy 1, policy_version 81900 (0.0009) -[2023-10-15 18:02:46,578][52866] Updated weights for policy 1, policy_version 81910 (0.0008) -[2023-10-15 18:02:46,942][52866] Updated weights for policy 1, policy_version 81920 (0.0008) -[2023-10-15 18:02:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167510016. Throughput: 0: 1798.0, 1: 1807.7. Samples: 41881252. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:02:48,441][51532] Avg episode reward: [(0, '77.190'), (1, '65.060')] -[2023-10-15 18:02:48,442][52410] Saving new best policy, reward=77.190! -[2023-10-15 18:02:49,518][52833] Updated weights for policy 0, policy_version 81670 (0.0008) -[2023-10-15 18:02:49,881][52833] Updated weights for policy 0, policy_version 81680 (0.0008) -[2023-10-15 18:02:50,255][52833] Updated weights for policy 0, policy_version 81690 (0.0009) -[2023-10-15 18:02:50,556][52866] Updated weights for policy 1, policy_version 81930 (0.0007) -[2023-10-15 18:02:50,921][52866] Updated weights for policy 1, policy_version 81940 (0.0008) -[2023-10-15 18:02:51,295][52866] Updated weights for policy 1, policy_version 81950 (0.0010) -[2023-10-15 18:02:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167575552. Throughput: 0: 1788.4, 1: 1790.8. Samples: 41902324. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:02:53,442][51532] Avg episode reward: [(0, '76.920'), (1, '65.630')] -[2023-10-15 18:02:53,914][52833] Updated weights for policy 0, policy_version 81700 (0.0009) -[2023-10-15 18:02:54,284][52833] Updated weights for policy 0, policy_version 81710 (0.0008) -[2023-10-15 18:02:54,648][52833] Updated weights for policy 0, policy_version 81720 (0.0008) -[2023-10-15 18:02:54,951][52866] Updated weights for policy 1, policy_version 81960 (0.0010) -[2023-10-15 18:02:55,314][52866] Updated weights for policy 1, policy_version 81970 (0.0010) -[2023-10-15 18:02:55,691][52866] Updated weights for policy 1, policy_version 81980 (0.0009) -[2023-10-15 18:02:58,397][52833] Updated weights for policy 0, policy_version 81730 (0.0007) -[2023-10-15 18:02:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167641088. Throughput: 0: 1800.5, 1: 1793.7. Samples: 41924980. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:02:58,441][51532] Avg episode reward: [(0, '74.170'), (1, '65.720')] -[2023-10-15 18:02:58,811][52833] Updated weights for policy 0, policy_version 81740 (0.0010) -[2023-10-15 18:02:59,173][52833] Updated weights for policy 0, policy_version 81750 (0.0009) -[2023-10-15 18:02:59,438][52866] Updated weights for policy 1, policy_version 81990 (0.0009) -[2023-10-15 18:02:59,534][52833] Updated weights for policy 0, policy_version 81760 (0.0008) -[2023-10-15 18:02:59,803][52866] Updated weights for policy 1, policy_version 82000 (0.0009) -[2023-10-15 18:03:00,171][52866] Updated weights for policy 1, policy_version 82010 (0.0008) -[2023-10-15 18:03:03,416][52833] Updated weights for policy 0, policy_version 81770 (0.0007) -[2023-10-15 18:03:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 167706624. Throughput: 0: 1794.0, 1: 1796.5. Samples: 41934840. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:03:03,441][51532] Avg episode reward: [(0, '73.610'), (1, '66.060')] -[2023-10-15 18:03:03,782][52833] Updated weights for policy 0, policy_version 81780 (0.0008) -[2023-10-15 18:03:03,847][52866] Updated weights for policy 1, policy_version 82020 (0.0007) -[2023-10-15 18:03:04,149][52833] Updated weights for policy 0, policy_version 81790 (0.0007) -[2023-10-15 18:03:04,215][52866] Updated weights for policy 1, policy_version 82030 (0.0007) -[2023-10-15 18:03:04,584][52866] Updated weights for policy 1, policy_version 82040 (0.0009) -[2023-10-15 18:03:07,868][52833] Updated weights for policy 0, policy_version 81800 (0.0008) -[2023-10-15 18:03:08,232][52833] Updated weights for policy 0, policy_version 81810 (0.0009) -[2023-10-15 18:03:08,259][52866] Updated weights for policy 1, policy_version 82050 (0.0010) -[2023-10-15 18:03:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 167772160. Throughput: 0: 1793.6, 1: 1804.4. Samples: 41957298. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:03:08,441][51532] Avg episode reward: [(0, '74.700'), (1, '66.390')] -[2023-10-15 18:03:08,603][52833] Updated weights for policy 0, policy_version 81820 (0.0007) -[2023-10-15 18:03:08,630][52866] Updated weights for policy 1, policy_version 82060 (0.0008) -[2023-10-15 18:03:08,999][52866] Updated weights for policy 1, policy_version 82070 (0.0009) -[2023-10-15 18:03:09,354][52866] Updated weights for policy 1, policy_version 82080 (0.0011) -[2023-10-15 18:03:12,245][52833] Updated weights for policy 0, policy_version 81830 (0.0007) -[2023-10-15 18:03:12,608][52833] Updated weights for policy 0, policy_version 81840 (0.0007) -[2023-10-15 18:03:12,981][52833] Updated weights for policy 0, policy_version 81850 (0.0007) -[2023-10-15 18:03:13,217][52866] Updated weights for policy 1, policy_version 82090 (0.0007) -[2023-10-15 18:03:13,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 167870464. Throughput: 0: 1801.3, 1: 1807.9. Samples: 41978698. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:03:13,442][51532] Avg episode reward: [(0, '74.650'), (1, '61.820')] -[2023-10-15 18:03:13,582][52866] Updated weights for policy 1, policy_version 82100 (0.0008) -[2023-10-15 18:03:13,944][52866] Updated weights for policy 1, policy_version 82110 (0.0007) -[2023-10-15 18:03:16,720][52833] Updated weights for policy 0, policy_version 81860 (0.0007) -[2023-10-15 18:03:17,095][52833] Updated weights for policy 0, policy_version 81870 (0.0007) -[2023-10-15 18:03:17,454][52833] Updated weights for policy 0, policy_version 81880 (0.0008) -[2023-10-15 18:03:17,594][52866] Updated weights for policy 1, policy_version 82120 (0.0009) -[2023-10-15 18:03:17,960][52866] Updated weights for policy 1, policy_version 82130 (0.0008) -[2023-10-15 18:03:18,325][52866] Updated weights for policy 1, policy_version 82140 (0.0007) -[2023-10-15 18:03:18,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 167936000. Throughput: 0: 1786.7, 1: 1800.5. Samples: 41989772. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:03:18,441][51532] Avg episode reward: [(0, '77.050'), (1, '61.020')] -[2023-10-15 18:03:21,347][52833] Updated weights for policy 0, policy_version 81890 (0.0007) -[2023-10-15 18:03:21,708][52833] Updated weights for policy 0, policy_version 81900 (0.0009) -[2023-10-15 18:03:22,025][52866] Updated weights for policy 1, policy_version 82150 (0.0010) -[2023-10-15 18:03:22,071][52833] Updated weights for policy 0, policy_version 81910 (0.0009) -[2023-10-15 18:03:22,395][52866] Updated weights for policy 1, policy_version 82160 (0.0007) -[2023-10-15 18:03:22,437][52833] Updated weights for policy 0, policy_version 81920 (0.0008) -[2023-10-15 18:03:22,752][52866] Updated weights for policy 1, policy_version 82170 (0.0007) -[2023-10-15 18:03:23,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 168034304. Throughput: 0: 1796.3, 1: 1806.1. Samples: 42011298. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:03:23,442][51532] Avg episode reward: [(0, '78.730'), (1, '62.460')] -[2023-10-15 18:03:23,442][52410] Saving new best policy, reward=78.730! -[2023-10-15 18:03:26,249][52833] Updated weights for policy 0, policy_version 81930 (0.0008) -[2023-10-15 18:03:26,403][52866] Updated weights for policy 1, policy_version 82180 (0.0009) -[2023-10-15 18:03:26,610][52833] Updated weights for policy 0, policy_version 81940 (0.0007) -[2023-10-15 18:03:26,773][52866] Updated weights for policy 1, policy_version 82190 (0.0007) -[2023-10-15 18:03:26,967][52833] Updated weights for policy 0, policy_version 81950 (0.0009) -[2023-10-15 18:03:27,147][52866] Updated weights for policy 1, policy_version 82200 (0.0010) -[2023-10-15 18:03:28,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 168099840. Throughput: 0: 1784.9, 1: 1802.3. Samples: 42031922. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:03:28,442][51532] Avg episode reward: [(0, '80.090'), (1, '61.880')] -[2023-10-15 18:03:28,455][52410] Saving new best policy, reward=80.090! -[2023-10-15 18:03:30,637][52833] Updated weights for policy 0, policy_version 81960 (0.0008) -[2023-10-15 18:03:30,810][52866] Updated weights for policy 1, policy_version 82210 (0.0009) -[2023-10-15 18:03:31,010][52833] Updated weights for policy 0, policy_version 81970 (0.0007) -[2023-10-15 18:03:31,174][52866] Updated weights for policy 1, policy_version 82220 (0.0007) -[2023-10-15 18:03:31,373][52833] Updated weights for policy 0, policy_version 81980 (0.0007) -[2023-10-15 18:03:31,535][52866] Updated weights for policy 1, policy_version 82230 (0.0008) -[2023-10-15 18:03:31,904][52866] Updated weights for policy 1, policy_version 82240 (0.0007) -[2023-10-15 18:03:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 168165376. Throughput: 0: 1805.6, 1: 1805.8. Samples: 42043768. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:03:33,442][51532] Avg episode reward: [(0, '78.850'), (1, '65.860')] -[2023-10-15 18:03:35,118][52833] Updated weights for policy 0, policy_version 81990 (0.0008) -[2023-10-15 18:03:35,479][52833] Updated weights for policy 0, policy_version 82000 (0.0009) -[2023-10-15 18:03:35,531][52866] Updated weights for policy 1, policy_version 82250 (0.0008) -[2023-10-15 18:03:35,841][52833] Updated weights for policy 0, policy_version 82010 (0.0008) -[2023-10-15 18:03:35,898][52866] Updated weights for policy 1, policy_version 82260 (0.0009) -[2023-10-15 18:03:36,257][52866] Updated weights for policy 1, policy_version 82270 (0.0008) -[2023-10-15 18:03:38,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 168230912. Throughput: 0: 1792.7, 1: 1811.5. Samples: 42064512. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) -[2023-10-15 18:03:38,441][51532] Avg episode reward: [(0, '80.420'), (1, '64.820')] -[2023-10-15 18:03:38,442][52410] Saving new best policy, reward=80.420! -[2023-10-15 18:03:39,508][52833] Updated weights for policy 0, policy_version 82020 (0.0008) -[2023-10-15 18:03:39,888][52833] Updated weights for policy 0, policy_version 82030 (0.0009) -[2023-10-15 18:03:40,086][52866] Updated weights for policy 1, policy_version 82280 (0.0009) -[2023-10-15 18:03:40,261][52833] Updated weights for policy 0, policy_version 82040 (0.0009) -[2023-10-15 18:03:40,452][52866] Updated weights for policy 1, policy_version 82290 (0.0008) -[2023-10-15 18:03:40,808][52866] Updated weights for policy 1, policy_version 82300 (0.0008) -[2023-10-15 18:03:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 168296448. Throughput: 0: 1793.4, 1: 1807.3. Samples: 42087016. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:03:43,442][51532] Avg episode reward: [(0, '82.310'), (1, '64.740')] -[2023-10-15 18:03:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000082048_84017152.pth... -[2023-10-15 18:03:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000082304_84279296.pth... -[2023-10-15 18:03:43,488][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000080384_82313216.pth -[2023-10-15 18:03:43,490][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000080608_82542592.pth -[2023-10-15 18:03:43,492][52410] Saving new best policy, reward=82.310! -[2023-10-15 18:03:44,095][52833] Updated weights for policy 0, policy_version 82050 (0.0009) -[2023-10-15 18:03:44,489][52833] Updated weights for policy 0, policy_version 82060 (0.0009) -[2023-10-15 18:03:44,662][52866] Updated weights for policy 1, policy_version 82310 (0.0007) -[2023-10-15 18:03:44,860][52833] Updated weights for policy 0, policy_version 82070 (0.0008) -[2023-10-15 18:03:45,024][52866] Updated weights for policy 1, policy_version 82320 (0.0008) -[2023-10-15 18:03:45,223][52833] Updated weights for policy 0, policy_version 82080 (0.0009) -[2023-10-15 18:03:45,385][52866] Updated weights for policy 1, policy_version 82330 (0.0010) -[2023-10-15 18:03:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 168361984. Throughput: 0: 1796.3, 1: 1802.0. Samples: 42096764. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:03:48,442][51532] Avg episode reward: [(0, '79.680'), (1, '61.690')] -[2023-10-15 18:03:48,943][52833] Updated weights for policy 0, policy_version 82090 (0.0009) -[2023-10-15 18:03:49,054][52866] Updated weights for policy 1, policy_version 82340 (0.0009) -[2023-10-15 18:03:49,304][52833] Updated weights for policy 0, policy_version 82100 (0.0009) -[2023-10-15 18:03:49,416][52866] Updated weights for policy 1, policy_version 82350 (0.0008) -[2023-10-15 18:03:49,675][52833] Updated weights for policy 0, policy_version 82110 (0.0009) -[2023-10-15 18:03:49,772][52866] Updated weights for policy 1, policy_version 82360 (0.0009) -[2023-10-15 18:03:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 168427520. Throughput: 0: 1797.2, 1: 1803.0. Samples: 42119306. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:03:53,441][51532] Avg episode reward: [(0, '80.780'), (1, '62.640')] -[2023-10-15 18:03:53,454][52833] Updated weights for policy 0, policy_version 82120 (0.0008) -[2023-10-15 18:03:53,578][52866] Updated weights for policy 1, policy_version 82370 (0.0008) -[2023-10-15 18:03:53,818][52833] Updated weights for policy 0, policy_version 82130 (0.0009) -[2023-10-15 18:03:53,983][52866] Updated weights for policy 1, policy_version 82380 (0.0007) -[2023-10-15 18:03:54,185][52833] Updated weights for policy 0, policy_version 82140 (0.0008) -[2023-10-15 18:03:54,356][52866] Updated weights for policy 1, policy_version 82390 (0.0009) -[2023-10-15 18:03:54,716][52866] Updated weights for policy 1, policy_version 82400 (0.0011) -[2023-10-15 18:03:57,691][52833] Updated weights for policy 0, policy_version 82150 (0.0009) -[2023-10-15 18:03:58,065][52833] Updated weights for policy 0, policy_version 82160 (0.0008) -[2023-10-15 18:03:58,438][52833] Updated weights for policy 0, policy_version 82170 (0.0007) -[2023-10-15 18:03:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 168493056. Throughput: 0: 1802.9, 1: 1805.9. Samples: 42141092. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:03:58,441][51532] Avg episode reward: [(0, '81.950'), (1, '62.420')] -[2023-10-15 18:03:58,534][52866] Updated weights for policy 1, policy_version 82410 (0.0007) -[2023-10-15 18:03:58,894][52866] Updated weights for policy 1, policy_version 82420 (0.0010) -[2023-10-15 18:03:59,270][52866] Updated weights for policy 1, policy_version 82430 (0.0011) -[2023-10-15 18:04:02,121][52833] Updated weights for policy 0, policy_version 82180 (0.0007) -[2023-10-15 18:04:02,477][52833] Updated weights for policy 0, policy_version 82190 (0.0007) -[2023-10-15 18:04:02,852][52833] Updated weights for policy 0, policy_version 82200 (0.0008) -[2023-10-15 18:04:03,103][52866] Updated weights for policy 1, policy_version 82440 (0.0009) -[2023-10-15 18:04:03,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 168591360. Throughput: 0: 1791.1, 1: 1800.5. Samples: 42151392. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:04:03,442][51532] Avg episode reward: [(0, '81.300'), (1, '60.750')] -[2023-10-15 18:04:03,471][52866] Updated weights for policy 1, policy_version 82450 (0.0008) -[2023-10-15 18:04:03,843][52866] Updated weights for policy 1, policy_version 82460 (0.0008) -[2023-10-15 18:04:06,485][52833] Updated weights for policy 0, policy_version 82210 (0.0008) -[2023-10-15 18:04:06,867][52833] Updated weights for policy 0, policy_version 82220 (0.0008) -[2023-10-15 18:04:07,232][52833] Updated weights for policy 0, policy_version 82230 (0.0010) -[2023-10-15 18:04:07,565][52866] Updated weights for policy 1, policy_version 82470 (0.0009) -[2023-10-15 18:04:07,594][52833] Updated weights for policy 0, policy_version 82240 (0.0007) -[2023-10-15 18:04:07,928][52866] Updated weights for policy 1, policy_version 82480 (0.0008) -[2023-10-15 18:04:08,289][52866] Updated weights for policy 1, policy_version 82490 (0.0008) -[2023-10-15 18:04:08,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 168656896. Throughput: 0: 1802.3, 1: 1802.3. Samples: 42173508. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:04:08,442][51532] Avg episode reward: [(0, '84.050'), (1, '61.520')] -[2023-10-15 18:04:08,443][52410] Saving new best policy, reward=84.050! -[2023-10-15 18:04:11,435][52833] Updated weights for policy 0, policy_version 82250 (0.0010) -[2023-10-15 18:04:11,801][52833] Updated weights for policy 0, policy_version 82260 (0.0009) -[2023-10-15 18:04:11,955][52866] Updated weights for policy 1, policy_version 82500 (0.0009) -[2023-10-15 18:04:12,170][52833] Updated weights for policy 0, policy_version 82270 (0.0007) -[2023-10-15 18:04:12,325][52866] Updated weights for policy 1, policy_version 82510 (0.0007) -[2023-10-15 18:04:12,690][52866] Updated weights for policy 1, policy_version 82520 (0.0007) -[2023-10-15 18:04:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 168755200. Throughput: 0: 1797.3, 1: 1800.2. Samples: 42193810. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:04:13,442][51532] Avg episode reward: [(0, '85.270'), (1, '66.870')] -[2023-10-15 18:04:13,451][52410] Saving new best policy, reward=85.270! -[2023-10-15 18:04:16,003][52833] Updated weights for policy 0, policy_version 82280 (0.0009) -[2023-10-15 18:04:16,359][52833] Updated weights for policy 0, policy_version 82290 (0.0009) -[2023-10-15 18:04:16,427][52866] Updated weights for policy 1, policy_version 82530 (0.0008) -[2023-10-15 18:04:16,732][52833] Updated weights for policy 0, policy_version 82300 (0.0008) -[2023-10-15 18:04:16,795][52866] Updated weights for policy 1, policy_version 82540 (0.0010) -[2023-10-15 18:04:17,148][52866] Updated weights for policy 1, policy_version 82550 (0.0008) -[2023-10-15 18:04:17,512][52866] Updated weights for policy 1, policy_version 82560 (0.0009) -[2023-10-15 18:04:18,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 168820736. Throughput: 0: 1803.7, 1: 1807.5. Samples: 42206268. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:04:18,442][51532] Avg episode reward: [(0, '85.010'), (1, '68.760')] -[2023-10-15 18:04:20,459][52833] Updated weights for policy 0, policy_version 82310 (0.0009) -[2023-10-15 18:04:20,830][52833] Updated weights for policy 0, policy_version 82320 (0.0009) -[2023-10-15 18:04:21,209][52833] Updated weights for policy 0, policy_version 82330 (0.0008) -[2023-10-15 18:04:21,329][52866] Updated weights for policy 1, policy_version 82570 (0.0009) -[2023-10-15 18:04:21,690][52866] Updated weights for policy 1, policy_version 82580 (0.0009) -[2023-10-15 18:04:22,060][52866] Updated weights for policy 1, policy_version 82590 (0.0007) -[2023-10-15 18:04:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 168886272. Throughput: 0: 1791.7, 1: 1802.9. Samples: 42226270. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:04:23,442][51532] Avg episode reward: [(0, '84.450'), (1, '68.390')] -[2023-10-15 18:04:25,151][52833] Updated weights for policy 0, policy_version 82340 (0.0008) -[2023-10-15 18:04:25,519][52833] Updated weights for policy 0, policy_version 82350 (0.0007) -[2023-10-15 18:04:25,826][52866] Updated weights for policy 1, policy_version 82600 (0.0009) -[2023-10-15 18:04:25,887][52833] Updated weights for policy 0, policy_version 82360 (0.0010) -[2023-10-15 18:04:26,192][52866] Updated weights for policy 1, policy_version 82610 (0.0009) -[2023-10-15 18:04:26,559][52866] Updated weights for policy 1, policy_version 82620 (0.0009) -[2023-10-15 18:04:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 168951808. Throughput: 0: 1785.8, 1: 1801.2. Samples: 42248432. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:04:28,441][51532] Avg episode reward: [(0, '82.640'), (1, '71.850')] -[2023-10-15 18:04:29,576][52833] Updated weights for policy 0, policy_version 82370 (0.0007) -[2023-10-15 18:04:29,961][52833] Updated weights for policy 0, policy_version 82380 (0.0009) -[2023-10-15 18:04:30,331][52833] Updated weights for policy 0, policy_version 82390 (0.0009) -[2023-10-15 18:04:30,387][52866] Updated weights for policy 1, policy_version 82630 (0.0008) -[2023-10-15 18:04:30,706][52833] Updated weights for policy 0, policy_version 82400 (0.0007) -[2023-10-15 18:04:30,753][52866] Updated weights for policy 1, policy_version 82640 (0.0008) -[2023-10-15 18:04:31,124][52866] Updated weights for policy 1, policy_version 82650 (0.0008) -[2023-10-15 18:04:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169017344. Throughput: 0: 1785.0, 1: 1814.5. Samples: 42258744. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-15 18:04:33,442][51532] Avg episode reward: [(0, '80.810'), (1, '70.460')] -[2023-10-15 18:04:34,433][52833] Updated weights for policy 0, policy_version 82410 (0.0009) -[2023-10-15 18:04:34,803][52833] Updated weights for policy 0, policy_version 82420 (0.0008) -[2023-10-15 18:04:34,900][52866] Updated weights for policy 1, policy_version 82660 (0.0009) -[2023-10-15 18:04:35,165][52833] Updated weights for policy 0, policy_version 82430 (0.0009) -[2023-10-15 18:04:35,268][52866] Updated weights for policy 1, policy_version 82670 (0.0008) -[2023-10-15 18:04:35,636][52866] Updated weights for policy 1, policy_version 82680 (0.0007) -[2023-10-15 18:04:38,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 169082880. Throughput: 0: 1790.8, 1: 1797.5. Samples: 42280780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:04:38,442][51532] Avg episode reward: [(0, '80.140'), (1, '70.300')] -[2023-10-15 18:04:39,047][52833] Updated weights for policy 0, policy_version 82440 (0.0008) -[2023-10-15 18:04:39,267][52866] Updated weights for policy 1, policy_version 82690 (0.0008) -[2023-10-15 18:04:39,413][52833] Updated weights for policy 0, policy_version 82450 (0.0009) -[2023-10-15 18:04:39,680][52866] Updated weights for policy 1, policy_version 82700 (0.0008) -[2023-10-15 18:04:39,777][52833] Updated weights for policy 0, policy_version 82460 (0.0008) -[2023-10-15 18:04:40,045][52866] Updated weights for policy 1, policy_version 82710 (0.0008) -[2023-10-15 18:04:40,407][52866] Updated weights for policy 1, policy_version 82720 (0.0008) -[2023-10-15 18:04:43,357][52833] Updated weights for policy 0, policy_version 82470 (0.0009) -[2023-10-15 18:04:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 169148416. Throughput: 0: 1803.9, 1: 1800.7. Samples: 42303300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:04:43,442][51532] Avg episode reward: [(0, '79.820'), (1, '69.730')] -[2023-10-15 18:04:43,720][52833] Updated weights for policy 0, policy_version 82480 (0.0008) -[2023-10-15 18:04:44,043][52866] Updated weights for policy 1, policy_version 82730 (0.0007) -[2023-10-15 18:04:44,089][52833] Updated weights for policy 0, policy_version 82490 (0.0009) -[2023-10-15 18:04:44,409][52866] Updated weights for policy 1, policy_version 82740 (0.0007) -[2023-10-15 18:04:44,771][52866] Updated weights for policy 1, policy_version 82750 (0.0008) -[2023-10-15 18:04:47,714][52833] Updated weights for policy 0, policy_version 82500 (0.0008) -[2023-10-15 18:04:48,081][52833] Updated weights for policy 0, policy_version 82510 (0.0009) -[2023-10-15 18:04:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 169213952. Throughput: 0: 1793.5, 1: 1797.4. Samples: 42312982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:04:48,441][51532] Avg episode reward: [(0, '81.220'), (1, '67.090')] -[2023-10-15 18:04:48,452][52833] Updated weights for policy 0, policy_version 82520 (0.0009) -[2023-10-15 18:04:48,575][52866] Updated weights for policy 1, policy_version 82760 (0.0010) -[2023-10-15 18:04:48,937][52866] Updated weights for policy 1, policy_version 82770 (0.0008) -[2023-10-15 18:04:49,304][52866] Updated weights for policy 1, policy_version 82780 (0.0009) -[2023-10-15 18:04:52,297][52833] Updated weights for policy 0, policy_version 82530 (0.0009) -[2023-10-15 18:04:52,659][52833] Updated weights for policy 0, policy_version 82540 (0.0007) -[2023-10-15 18:04:53,008][52866] Updated weights for policy 1, policy_version 82790 (0.0007) -[2023-10-15 18:04:53,033][52833] Updated weights for policy 0, policy_version 82550 (0.0009) -[2023-10-15 18:04:53,367][52866] Updated weights for policy 1, policy_version 82800 (0.0007) -[2023-10-15 18:04:53,402][52833] Updated weights for policy 0, policy_version 82560 (0.0009) -[2023-10-15 18:04:53,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 169312256. Throughput: 0: 1804.9, 1: 1795.4. Samples: 42335522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:04:53,441][51532] Avg episode reward: [(0, '80.730'), (1, '69.160')] -[2023-10-15 18:04:53,739][52866] Updated weights for policy 1, policy_version 82810 (0.0009) -[2023-10-15 18:04:57,298][52833] Updated weights for policy 0, policy_version 82570 (0.0007) -[2023-10-15 18:04:57,499][52866] Updated weights for policy 1, policy_version 82820 (0.0008) -[2023-10-15 18:04:57,666][52833] Updated weights for policy 0, policy_version 82580 (0.0008) -[2023-10-15 18:04:57,855][52866] Updated weights for policy 1, policy_version 82830 (0.0008) -[2023-10-15 18:04:58,039][52833] Updated weights for policy 0, policy_version 82590 (0.0008) -[2023-10-15 18:04:58,226][52866] Updated weights for policy 1, policy_version 82840 (0.0007) -[2023-10-15 18:04:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 169377792. Throughput: 0: 1797.2, 1: 1805.6. Samples: 42355938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:04:58,441][51532] Avg episode reward: [(0, '83.600'), (1, '68.580')] -[2023-10-15 18:05:01,904][52833] Updated weights for policy 0, policy_version 82600 (0.0008) -[2023-10-15 18:05:01,947][52866] Updated weights for policy 1, policy_version 82850 (0.0008) -[2023-10-15 18:05:02,260][52833] Updated weights for policy 0, policy_version 82610 (0.0008) -[2023-10-15 18:05:02,307][52866] Updated weights for policy 1, policy_version 82860 (0.0007) -[2023-10-15 18:05:02,625][52833] Updated weights for policy 0, policy_version 82620 (0.0008) -[2023-10-15 18:05:02,678][52866] Updated weights for policy 1, policy_version 82870 (0.0008) -[2023-10-15 18:05:03,043][52866] Updated weights for policy 1, policy_version 82880 (0.0010) -[2023-10-15 18:05:03,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 169476096. Throughput: 0: 1793.8, 1: 1792.0. Samples: 42367630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:05:03,441][51532] Avg episode reward: [(0, '81.080'), (1, '68.790')] -[2023-10-15 18:05:06,287][52833] Updated weights for policy 0, policy_version 82630 (0.0009) -[2023-10-15 18:05:06,656][52833] Updated weights for policy 0, policy_version 82640 (0.0007) -[2023-10-15 18:05:06,816][52866] Updated weights for policy 1, policy_version 82890 (0.0008) -[2023-10-15 18:05:07,028][52833] Updated weights for policy 0, policy_version 82650 (0.0008) -[2023-10-15 18:05:07,178][52866] Updated weights for policy 1, policy_version 82900 (0.0007) -[2023-10-15 18:05:07,548][52866] Updated weights for policy 1, policy_version 82910 (0.0008) -[2023-10-15 18:05:08,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 169541632. Throughput: 0: 1806.3, 1: 1801.8. Samples: 42388632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:05:08,442][51532] Avg episode reward: [(0, '80.080'), (1, '69.530')] -[2023-10-15 18:05:10,792][52833] Updated weights for policy 0, policy_version 82660 (0.0008) -[2023-10-15 18:05:11,151][52833] Updated weights for policy 0, policy_version 82670 (0.0007) -[2023-10-15 18:05:11,351][52866] Updated weights for policy 1, policy_version 82920 (0.0008) -[2023-10-15 18:05:11,522][52833] Updated weights for policy 0, policy_version 82680 (0.0008) -[2023-10-15 18:05:11,708][52866] Updated weights for policy 1, policy_version 82930 (0.0008) -[2023-10-15 18:05:12,072][52866] Updated weights for policy 1, policy_version 82940 (0.0010) -[2023-10-15 18:05:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169607168. Throughput: 0: 1794.8, 1: 1787.0. Samples: 42409610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:05:13,442][51532] Avg episode reward: [(0, '75.480'), (1, '68.510')] -[2023-10-15 18:05:15,328][52833] Updated weights for policy 0, policy_version 82690 (0.0008) -[2023-10-15 18:05:15,729][52833] Updated weights for policy 0, policy_version 82700 (0.0009) -[2023-10-15 18:05:15,875][52866] Updated weights for policy 1, policy_version 82950 (0.0010) -[2023-10-15 18:05:16,087][52833] Updated weights for policy 0, policy_version 82710 (0.0007) -[2023-10-15 18:05:16,245][52866] Updated weights for policy 1, policy_version 82960 (0.0007) -[2023-10-15 18:05:16,456][52833] Updated weights for policy 0, policy_version 82720 (0.0007) -[2023-10-15 18:05:16,611][52866] Updated weights for policy 1, policy_version 82970 (0.0008) -[2023-10-15 18:05:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 169672704. Throughput: 0: 1810.2, 1: 1798.0. Samples: 42421110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:05:18,442][51532] Avg episode reward: [(0, '77.120'), (1, '70.260')] -[2023-10-15 18:05:20,280][52833] Updated weights for policy 0, policy_version 82730 (0.0009) -[2023-10-15 18:05:20,483][52866] Updated weights for policy 1, policy_version 82980 (0.0008) -[2023-10-15 18:05:20,646][52833] Updated weights for policy 0, policy_version 82740 (0.0007) -[2023-10-15 18:05:20,849][52866] Updated weights for policy 1, policy_version 82990 (0.0007) -[2023-10-15 18:05:21,011][52833] Updated weights for policy 0, policy_version 82750 (0.0008) -[2023-10-15 18:05:21,222][52866] Updated weights for policy 1, policy_version 83000 (0.0009) -[2023-10-15 18:05:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169738240. Throughput: 0: 1785.8, 1: 1784.6. Samples: 42441446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:05:23,442][51532] Avg episode reward: [(0, '77.300'), (1, '68.400')] -[2023-10-15 18:05:24,698][52833] Updated weights for policy 0, policy_version 82760 (0.0008) -[2023-10-15 18:05:25,014][52866] Updated weights for policy 1, policy_version 83010 (0.0010) -[2023-10-15 18:05:25,066][52833] Updated weights for policy 0, policy_version 82770 (0.0010) -[2023-10-15 18:05:25,408][52866] Updated weights for policy 1, policy_version 83020 (0.0008) -[2023-10-15 18:05:25,436][52833] Updated weights for policy 0, policy_version 82780 (0.0007) -[2023-10-15 18:05:25,778][52866] Updated weights for policy 1, policy_version 83030 (0.0009) -[2023-10-15 18:05:26,143][52866] Updated weights for policy 1, policy_version 83040 (0.0009) -[2023-10-15 18:05:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 169803776. Throughput: 0: 1790.5, 1: 1785.7. Samples: 42464230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:05:28,442][51532] Avg episode reward: [(0, '77.850'), (1, '65.850')] -[2023-10-15 18:05:29,207][52833] Updated weights for policy 0, policy_version 82790 (0.0007) -[2023-10-15 18:05:29,568][52833] Updated weights for policy 0, policy_version 82800 (0.0010) -[2023-10-15 18:05:29,827][52866] Updated weights for policy 1, policy_version 83050 (0.0008) -[2023-10-15 18:05:29,944][52833] Updated weights for policy 0, policy_version 82810 (0.0009) -[2023-10-15 18:05:30,206][52866] Updated weights for policy 1, policy_version 83060 (0.0009) -[2023-10-15 18:05:30,577][52866] Updated weights for policy 1, policy_version 83070 (0.0009) -[2023-10-15 18:05:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 169869312. Throughput: 0: 1790.3, 1: 1789.5. Samples: 42474070. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:05:33,441][51532] Avg episode reward: [(0, '80.430'), (1, '63.500')] -[2023-10-15 18:05:33,654][52833] Updated weights for policy 0, policy_version 82820 (0.0007) -[2023-10-15 18:05:34,018][52833] Updated weights for policy 0, policy_version 82830 (0.0010) -[2023-10-15 18:05:34,173][52866] Updated weights for policy 1, policy_version 83080 (0.0007) -[2023-10-15 18:05:34,392][52833] Updated weights for policy 0, policy_version 82840 (0.0009) -[2023-10-15 18:05:34,539][52866] Updated weights for policy 1, policy_version 83090 (0.0008) -[2023-10-15 18:05:34,898][52866] Updated weights for policy 1, policy_version 83100 (0.0009) -[2023-10-15 18:05:38,068][52833] Updated weights for policy 0, policy_version 82850 (0.0009) -[2023-10-15 18:05:38,430][52833] Updated weights for policy 0, policy_version 82860 (0.0011) -[2023-10-15 18:05:38,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 169934848. Throughput: 0: 1785.2, 1: 1790.7. Samples: 42496442. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:05:38,443][51532] Avg episode reward: [(0, '78.960'), (1, '66.120')] -[2023-10-15 18:05:38,804][52833] Updated weights for policy 0, policy_version 82870 (0.0007) -[2023-10-15 18:05:38,855][52866] Updated weights for policy 1, policy_version 83110 (0.0009) -[2023-10-15 18:05:39,175][52833] Updated weights for policy 0, policy_version 82880 (0.0008) -[2023-10-15 18:05:39,215][52866] Updated weights for policy 1, policy_version 83120 (0.0008) -[2023-10-15 18:05:39,581][52866] Updated weights for policy 1, policy_version 83130 (0.0007) -[2023-10-15 18:05:42,948][52833] Updated weights for policy 0, policy_version 82890 (0.0007) -[2023-10-15 18:05:43,233][52866] Updated weights for policy 1, policy_version 83140 (0.0008) -[2023-10-15 18:05:43,324][52833] Updated weights for policy 0, policy_version 82900 (0.0007) -[2023-10-15 18:05:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 170000384. Throughput: 0: 1801.7, 1: 1809.6. Samples: 42518446. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:05:43,442][51532] Avg episode reward: [(0, '76.810'), (1, '67.050')] -[2023-10-15 18:05:43,596][52866] Updated weights for policy 1, policy_version 83150 (0.0007) -[2023-10-15 18:05:43,677][52833] Updated weights for policy 0, policy_version 82910 (0.0008) -[2023-10-15 18:05:43,748][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000082912_84901888.pth... -[2023-10-15 18:05:43,780][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000081216_83165184.pth -[2023-10-15 18:05:43,959][52866] Updated weights for policy 1, policy_version 83160 (0.0008) -[2023-10-15 18:05:44,253][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000083168_85164032.pth... -[2023-10-15 18:05:44,282][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000081472_83427328.pth -[2023-10-15 18:05:47,347][52833] Updated weights for policy 0, policy_version 82920 (0.0009) -[2023-10-15 18:05:47,714][52833] Updated weights for policy 0, policy_version 82930 (0.0010) -[2023-10-15 18:05:47,754][52866] Updated weights for policy 1, policy_version 83170 (0.0008) -[2023-10-15 18:05:48,082][52833] Updated weights for policy 0, policy_version 82940 (0.0008) -[2023-10-15 18:05:48,121][52866] Updated weights for policy 1, policy_version 83180 (0.0008) -[2023-10-15 18:05:48,441][51532] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 170098688. Throughput: 0: 1786.7, 1: 1787.4. Samples: 42528464. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:05:48,441][51532] Avg episode reward: [(0, '74.590'), (1, '69.320')] -[2023-10-15 18:05:48,495][52866] Updated weights for policy 1, policy_version 83190 (0.0010) -[2023-10-15 18:05:48,858][52866] Updated weights for policy 1, policy_version 83200 (0.0008) -[2023-10-15 18:05:51,775][52833] Updated weights for policy 0, policy_version 82950 (0.0008) -[2023-10-15 18:05:52,142][52833] Updated weights for policy 0, policy_version 82960 (0.0008) -[2023-10-15 18:05:52,513][52833] Updated weights for policy 0, policy_version 82970 (0.0008) -[2023-10-15 18:05:52,694][52866] Updated weights for policy 1, policy_version 83210 (0.0007) -[2023-10-15 18:05:53,062][52866] Updated weights for policy 1, policy_version 83220 (0.0011) -[2023-10-15 18:05:53,438][52866] Updated weights for policy 1, policy_version 83230 (0.0009) -[2023-10-15 18:05:53,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 170164224. Throughput: 0: 1795.9, 1: 1803.6. Samples: 42550608. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:05:53,442][51532] Avg episode reward: [(0, '74.770'), (1, '72.330')] -[2023-10-15 18:05:56,262][52833] Updated weights for policy 0, policy_version 82980 (0.0007) -[2023-10-15 18:05:56,631][52833] Updated weights for policy 0, policy_version 82990 (0.0009) -[2023-10-15 18:05:56,997][52833] Updated weights for policy 0, policy_version 83000 (0.0009) -[2023-10-15 18:05:57,324][52866] Updated weights for policy 1, policy_version 83240 (0.0008) -[2023-10-15 18:05:57,695][52866] Updated weights for policy 1, policy_version 83250 (0.0007) -[2023-10-15 18:05:58,058][52866] Updated weights for policy 1, policy_version 83260 (0.0008) -[2023-10-15 18:05:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 170262528. Throughput: 0: 1785.6, 1: 1794.0. Samples: 42570692. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:05:58,441][51532] Avg episode reward: [(0, '72.610'), (1, '71.430')] -[2023-10-15 18:06:00,741][52833] Updated weights for policy 0, policy_version 83010 (0.0008) -[2023-10-15 18:06:01,129][52833] Updated weights for policy 0, policy_version 83020 (0.0010) -[2023-10-15 18:06:01,499][52833] Updated weights for policy 0, policy_version 83030 (0.0009) -[2023-10-15 18:06:01,809][52866] Updated weights for policy 1, policy_version 83270 (0.0009) -[2023-10-15 18:06:01,865][52833] Updated weights for policy 0, policy_version 83040 (0.0007) -[2023-10-15 18:06:02,175][52866] Updated weights for policy 1, policy_version 83280 (0.0009) -[2023-10-15 18:06:02,544][52866] Updated weights for policy 1, policy_version 83290 (0.0011) -[2023-10-15 18:06:03,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 170328064. Throughput: 0: 1801.1, 1: 1797.6. Samples: 42583052. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:06:03,442][51532] Avg episode reward: [(0, '71.280'), (1, '71.980')] -[2023-10-15 18:06:05,520][52833] Updated weights for policy 0, policy_version 83050 (0.0007) -[2023-10-15 18:06:05,895][52833] Updated weights for policy 0, policy_version 83060 (0.0008) -[2023-10-15 18:06:06,265][52833] Updated weights for policy 0, policy_version 83070 (0.0008) -[2023-10-15 18:06:06,346][52866] Updated weights for policy 1, policy_version 83300 (0.0010) -[2023-10-15 18:06:06,711][52866] Updated weights for policy 1, policy_version 83310 (0.0009) -[2023-10-15 18:06:07,077][52866] Updated weights for policy 1, policy_version 83320 (0.0007) -[2023-10-15 18:06:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170393600. Throughput: 0: 1798.9, 1: 1800.7. Samples: 42603430. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:06:08,442][51532] Avg episode reward: [(0, '71.570'), (1, '71.970')] -[2023-10-15 18:06:09,888][52833] Updated weights for policy 0, policy_version 83080 (0.0009) -[2023-10-15 18:06:10,255][52833] Updated weights for policy 0, policy_version 83090 (0.0010) -[2023-10-15 18:06:10,632][52833] Updated weights for policy 0, policy_version 83100 (0.0011) -[2023-10-15 18:06:10,997][52866] Updated weights for policy 1, policy_version 83330 (0.0009) -[2023-10-15 18:06:11,407][52866] Updated weights for policy 1, policy_version 83340 (0.0008) -[2023-10-15 18:06:11,782][52866] Updated weights for policy 1, policy_version 83350 (0.0010) -[2023-10-15 18:06:12,149][52866] Updated weights for policy 1, policy_version 83360 (0.0010) -[2023-10-15 18:06:13,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 170459136. Throughput: 0: 1798.3, 1: 1783.4. Samples: 42625404. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:06:13,441][51532] Avg episode reward: [(0, '69.760'), (1, '68.600')] -[2023-10-15 18:06:14,329][52833] Updated weights for policy 0, policy_version 83110 (0.0008) -[2023-10-15 18:06:14,704][52833] Updated weights for policy 0, policy_version 83120 (0.0008) -[2023-10-15 18:06:15,084][52833] Updated weights for policy 0, policy_version 83130 (0.0011) -[2023-10-15 18:06:15,784][52866] Updated weights for policy 1, policy_version 83370 (0.0008) -[2023-10-15 18:06:16,154][52866] Updated weights for policy 1, policy_version 83380 (0.0007) -[2023-10-15 18:06:16,523][52866] Updated weights for policy 1, policy_version 83390 (0.0009) -[2023-10-15 18:06:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170524672. Throughput: 0: 1797.8, 1: 1800.2. Samples: 42635982. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:06:18,442][51532] Avg episode reward: [(0, '73.070'), (1, '65.650')] -[2023-10-15 18:06:18,839][52833] Updated weights for policy 0, policy_version 83140 (0.0008) -[2023-10-15 18:06:19,205][52833] Updated weights for policy 0, policy_version 83150 (0.0008) -[2023-10-15 18:06:19,579][52833] Updated weights for policy 0, policy_version 83160 (0.0008) -[2023-10-15 18:06:20,186][52866] Updated weights for policy 1, policy_version 83400 (0.0008) -[2023-10-15 18:06:20,554][52866] Updated weights for policy 1, policy_version 83410 (0.0009) -[2023-10-15 18:06:20,917][52866] Updated weights for policy 1, policy_version 83420 (0.0007) -[2023-10-15 18:06:23,329][52833] Updated weights for policy 0, policy_version 83170 (0.0009) -[2023-10-15 18:06:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 170590208. Throughput: 0: 1802.8, 1: 1784.0. Samples: 42657848. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) -[2023-10-15 18:06:23,442][51532] Avg episode reward: [(0, '70.630'), (1, '65.950')] -[2023-10-15 18:06:23,710][52833] Updated weights for policy 0, policy_version 83180 (0.0008) -[2023-10-15 18:06:24,079][52833] Updated weights for policy 0, policy_version 83190 (0.0007) -[2023-10-15 18:06:24,444][52833] Updated weights for policy 0, policy_version 83200 (0.0008) -[2023-10-15 18:06:24,641][52866] Updated weights for policy 1, policy_version 83430 (0.0008) -[2023-10-15 18:06:25,016][52866] Updated weights for policy 1, policy_version 83440 (0.0011) -[2023-10-15 18:06:25,380][52866] Updated weights for policy 1, policy_version 83450 (0.0008) -[2023-10-15 18:06:28,142][52833] Updated weights for policy 0, policy_version 83210 (0.0010) -[2023-10-15 18:06:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 170655744. Throughput: 0: 1815.2, 1: 1781.6. Samples: 42680300. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:06:28,441][51532] Avg episode reward: [(0, '71.020'), (1, '66.930')] -[2023-10-15 18:06:28,504][52833] Updated weights for policy 0, policy_version 83220 (0.0008) -[2023-10-15 18:06:28,885][52833] Updated weights for policy 0, policy_version 83230 (0.0010) -[2023-10-15 18:06:29,136][52866] Updated weights for policy 1, policy_version 83460 (0.0008) -[2023-10-15 18:06:29,502][52866] Updated weights for policy 1, policy_version 83470 (0.0009) -[2023-10-15 18:06:29,865][52866] Updated weights for policy 1, policy_version 83480 (0.0007) -[2023-10-15 18:06:32,646][52833] Updated weights for policy 0, policy_version 83240 (0.0007) -[2023-10-15 18:06:33,020][52833] Updated weights for policy 0, policy_version 83250 (0.0007) -[2023-10-15 18:06:33,403][52833] Updated weights for policy 0, policy_version 83260 (0.0008) -[2023-10-15 18:06:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 170721280. Throughput: 0: 1814.4, 1: 1781.6. Samples: 42690282. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:06:33,441][51532] Avg episode reward: [(0, '68.550'), (1, '67.310')] -[2023-10-15 18:06:33,605][52866] Updated weights for policy 1, policy_version 83490 (0.0008) -[2023-10-15 18:06:33,973][52866] Updated weights for policy 1, policy_version 83500 (0.0007) -[2023-10-15 18:06:34,343][52866] Updated weights for policy 1, policy_version 83510 (0.0007) -[2023-10-15 18:06:34,711][52866] Updated weights for policy 1, policy_version 83520 (0.0008) -[2023-10-15 18:06:37,128][52833] Updated weights for policy 0, policy_version 83270 (0.0007) -[2023-10-15 18:06:37,496][52833] Updated weights for policy 0, policy_version 83280 (0.0010) -[2023-10-15 18:06:37,868][52833] Updated weights for policy 0, policy_version 83290 (0.0007) -[2023-10-15 18:06:38,436][52866] Updated weights for policy 1, policy_version 83530 (0.0007) -[2023-10-15 18:06:38,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 170819584. Throughput: 0: 1823.1, 1: 1779.6. Samples: 42712726. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:06:38,441][51532] Avg episode reward: [(0, '74.250'), (1, '64.570')] -[2023-10-15 18:06:38,801][52866] Updated weights for policy 1, policy_version 83540 (0.0007) -[2023-10-15 18:06:39,167][52866] Updated weights for policy 1, policy_version 83550 (0.0007) -[2023-10-15 18:06:41,622][52833] Updated weights for policy 0, policy_version 83300 (0.0010) -[2023-10-15 18:06:41,988][52833] Updated weights for policy 0, policy_version 83310 (0.0011) -[2023-10-15 18:06:42,345][52833] Updated weights for policy 0, policy_version 83320 (0.0007) -[2023-10-15 18:06:42,961][52866] Updated weights for policy 1, policy_version 83560 (0.0008) -[2023-10-15 18:06:43,338][52866] Updated weights for policy 1, policy_version 83570 (0.0008) -[2023-10-15 18:06:43,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 170885120. Throughput: 0: 1815.7, 1: 1806.0. Samples: 42733672. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:06:43,442][51532] Avg episode reward: [(0, '74.300'), (1, '61.140')] -[2023-10-15 18:06:43,699][52866] Updated weights for policy 1, policy_version 83580 (0.0008) -[2023-10-15 18:06:45,934][52833] Updated weights for policy 0, policy_version 83330 (0.0007) -[2023-10-15 18:06:46,323][52833] Updated weights for policy 0, policy_version 83340 (0.0008) -[2023-10-15 18:06:46,691][52833] Updated weights for policy 0, policy_version 83350 (0.0007) -[2023-10-15 18:06:47,064][52833] Updated weights for policy 0, policy_version 83360 (0.0008) -[2023-10-15 18:06:47,347][52866] Updated weights for policy 1, policy_version 83590 (0.0007) -[2023-10-15 18:06:47,721][52866] Updated weights for policy 1, policy_version 83600 (0.0009) -[2023-10-15 18:06:48,087][52866] Updated weights for policy 1, policy_version 83610 (0.0010) -[2023-10-15 18:06:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 170983424. Throughput: 0: 1822.7, 1: 1788.9. Samples: 42745572. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:06:48,441][51532] Avg episode reward: [(0, '76.570'), (1, '57.770')] -[2023-10-15 18:06:50,710][52833] Updated weights for policy 0, policy_version 83370 (0.0008) -[2023-10-15 18:06:51,074][52833] Updated weights for policy 0, policy_version 83380 (0.0007) -[2023-10-15 18:06:51,438][52833] Updated weights for policy 0, policy_version 83390 (0.0007) -[2023-10-15 18:06:51,806][52866] Updated weights for policy 1, policy_version 83620 (0.0007) -[2023-10-15 18:06:52,177][52866] Updated weights for policy 1, policy_version 83630 (0.0008) -[2023-10-15 18:06:52,544][52866] Updated weights for policy 1, policy_version 83640 (0.0009) -[2023-10-15 18:06:53,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 171048960. Throughput: 0: 1812.3, 1: 1806.9. Samples: 42766294. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:06:53,442][51532] Avg episode reward: [(0, '74.840'), (1, '55.740')] -[2023-10-15 18:06:55,198][52833] Updated weights for policy 0, policy_version 83400 (0.0008) -[2023-10-15 18:06:55,566][52833] Updated weights for policy 0, policy_version 83410 (0.0010) -[2023-10-15 18:06:55,928][52833] Updated weights for policy 0, policy_version 83420 (0.0007) -[2023-10-15 18:06:56,079][52866] Updated weights for policy 1, policy_version 83650 (0.0008) -[2023-10-15 18:06:56,439][52866] Updated weights for policy 1, policy_version 83660 (0.0008) -[2023-10-15 18:06:56,803][52866] Updated weights for policy 1, policy_version 83670 (0.0008) -[2023-10-15 18:06:57,165][52866] Updated weights for policy 1, policy_version 83680 (0.0007) -[2023-10-15 18:06:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171114496. Throughput: 0: 1807.0, 1: 1803.7. Samples: 42787888. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:06:58,441][51532] Avg episode reward: [(0, '73.960'), (1, '54.300')] -[2023-10-15 18:06:59,531][52833] Updated weights for policy 0, policy_version 83430 (0.0008) -[2023-10-15 18:06:59,891][52833] Updated weights for policy 0, policy_version 83440 (0.0009) -[2023-10-15 18:07:00,263][52833] Updated weights for policy 0, policy_version 83450 (0.0010) -[2023-10-15 18:07:00,922][52866] Updated weights for policy 1, policy_version 83690 (0.0008) -[2023-10-15 18:07:01,283][52866] Updated weights for policy 1, policy_version 83700 (0.0008) -[2023-10-15 18:07:01,650][52866] Updated weights for policy 1, policy_version 83710 (0.0009) -[2023-10-15 18:07:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.2). Total num frames: 171180032. Throughput: 0: 1808.6, 1: 1806.7. Samples: 42798668. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:07:03,441][51532] Avg episode reward: [(0, '74.620'), (1, '56.400')] -[2023-10-15 18:07:03,985][52833] Updated weights for policy 0, policy_version 83460 (0.0007) -[2023-10-15 18:07:04,350][52833] Updated weights for policy 0, policy_version 83470 (0.0007) -[2023-10-15 18:07:04,723][52833] Updated weights for policy 0, policy_version 83480 (0.0008) -[2023-10-15 18:07:05,406][52866] Updated weights for policy 1, policy_version 83720 (0.0008) -[2023-10-15 18:07:05,781][52866] Updated weights for policy 1, policy_version 83730 (0.0009) -[2023-10-15 18:07:06,151][52866] Updated weights for policy 1, policy_version 83740 (0.0010) -[2023-10-15 18:07:08,351][52833] Updated weights for policy 0, policy_version 83490 (0.0008) -[2023-10-15 18:07:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171245568. Throughput: 0: 1811.2, 1: 1801.3. Samples: 42820412. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:07:08,442][51532] Avg episode reward: [(0, '74.040'), (1, '53.670')] -[2023-10-15 18:07:08,723][52833] Updated weights for policy 0, policy_version 83500 (0.0008) -[2023-10-15 18:07:09,094][52833] Updated weights for policy 0, policy_version 83510 (0.0009) -[2023-10-15 18:07:09,461][52833] Updated weights for policy 0, policy_version 83520 (0.0009) -[2023-10-15 18:07:09,833][52866] Updated weights for policy 1, policy_version 83750 (0.0009) -[2023-10-15 18:07:10,199][52866] Updated weights for policy 1, policy_version 83760 (0.0011) -[2023-10-15 18:07:10,566][52866] Updated weights for policy 1, policy_version 83770 (0.0010) -[2023-10-15 18:07:13,320][52833] Updated weights for policy 0, policy_version 83530 (0.0007) -[2023-10-15 18:07:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 171311104. Throughput: 0: 1810.2, 1: 1806.8. Samples: 42843068. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:07:13,442][51532] Avg episode reward: [(0, '74.770'), (1, '53.920')] -[2023-10-15 18:07:13,693][52833] Updated weights for policy 0, policy_version 83540 (0.0007) -[2023-10-15 18:07:14,054][52833] Updated weights for policy 0, policy_version 83550 (0.0009) -[2023-10-15 18:07:14,263][52866] Updated weights for policy 1, policy_version 83780 (0.0009) -[2023-10-15 18:07:14,641][52866] Updated weights for policy 1, policy_version 83790 (0.0008) -[2023-10-15 18:07:15,005][52866] Updated weights for policy 1, policy_version 83800 (0.0008) -[2023-10-15 18:07:17,755][52833] Updated weights for policy 0, policy_version 83560 (0.0008) -[2023-10-15 18:07:18,117][52833] Updated weights for policy 0, policy_version 83570 (0.0008) -[2023-10-15 18:07:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 171376640. Throughput: 0: 1807.7, 1: 1810.1. Samples: 42853084. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-15 18:07:18,441][51532] Avg episode reward: [(0, '72.940'), (1, '54.220')] -[2023-10-15 18:07:18,499][52833] Updated weights for policy 0, policy_version 83580 (0.0007) -[2023-10-15 18:07:18,718][52866] Updated weights for policy 1, policy_version 83810 (0.0008) -[2023-10-15 18:07:19,091][52866] Updated weights for policy 1, policy_version 83820 (0.0007) -[2023-10-15 18:07:19,466][52866] Updated weights for policy 1, policy_version 83830 (0.0007) -[2023-10-15 18:07:19,830][52866] Updated weights for policy 1, policy_version 83840 (0.0008) -[2023-10-15 18:07:22,326][52833] Updated weights for policy 0, policy_version 83590 (0.0009) -[2023-10-15 18:07:22,695][52833] Updated weights for policy 0, policy_version 83600 (0.0008) -[2023-10-15 18:07:23,058][52833] Updated weights for policy 0, policy_version 83610 (0.0010) -[2023-10-15 18:07:23,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 171474944. Throughput: 0: 1808.3, 1: 1816.9. Samples: 42875858. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:07:23,442][51532] Avg episode reward: [(0, '71.880'), (1, '52.940')] -[2023-10-15 18:07:23,533][52866] Updated weights for policy 1, policy_version 83850 (0.0008) -[2023-10-15 18:07:23,908][52866] Updated weights for policy 1, policy_version 83860 (0.0009) -[2023-10-15 18:07:24,268][52866] Updated weights for policy 1, policy_version 83870 (0.0008) -[2023-10-15 18:07:26,637][52833] Updated weights for policy 0, policy_version 83620 (0.0008) -[2023-10-15 18:07:27,009][52833] Updated weights for policy 0, policy_version 83630 (0.0008) -[2023-10-15 18:07:27,371][52833] Updated weights for policy 0, policy_version 83640 (0.0007) -[2023-10-15 18:07:27,903][52866] Updated weights for policy 1, policy_version 83880 (0.0008) -[2023-10-15 18:07:28,271][52866] Updated weights for policy 1, policy_version 83890 (0.0009) -[2023-10-15 18:07:28,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 171540480. Throughput: 0: 1808.8, 1: 1814.3. Samples: 42896714. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:07:28,442][51532] Avg episode reward: [(0, '73.490'), (1, '53.600')] -[2023-10-15 18:07:28,638][52866] Updated weights for policy 1, policy_version 83900 (0.0008) -[2023-10-15 18:07:31,219][52833] Updated weights for policy 0, policy_version 83650 (0.0008) -[2023-10-15 18:07:31,584][52833] Updated weights for policy 0, policy_version 83660 (0.0010) -[2023-10-15 18:07:31,945][52833] Updated weights for policy 0, policy_version 83670 (0.0007) -[2023-10-15 18:07:32,317][52833] Updated weights for policy 0, policy_version 83680 (0.0007) -[2023-10-15 18:07:32,409][52866] Updated weights for policy 1, policy_version 83910 (0.0008) -[2023-10-15 18:07:32,779][52866] Updated weights for policy 1, policy_version 83920 (0.0007) -[2023-10-15 18:07:33,156][52866] Updated weights for policy 1, policy_version 83930 (0.0007) -[2023-10-15 18:07:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 171638784. Throughput: 0: 1805.3, 1: 1813.0. Samples: 42908396. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:07:33,442][51532] Avg episode reward: [(0, '70.290'), (1, '55.220')] -[2023-10-15 18:07:36,229][52833] Updated weights for policy 0, policy_version 83690 (0.0009) -[2023-10-15 18:07:36,603][52833] Updated weights for policy 0, policy_version 83700 (0.0011) -[2023-10-15 18:07:36,932][52866] Updated weights for policy 1, policy_version 83940 (0.0008) -[2023-10-15 18:07:36,972][52833] Updated weights for policy 0, policy_version 83710 (0.0008) -[2023-10-15 18:07:37,298][52866] Updated weights for policy 1, policy_version 83950 (0.0010) -[2023-10-15 18:07:37,664][52866] Updated weights for policy 1, policy_version 83960 (0.0008) -[2023-10-15 18:07:38,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 171704320. Throughput: 0: 1809.7, 1: 1810.3. Samples: 42929196. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:07:38,442][51532] Avg episode reward: [(0, '68.210'), (1, '55.660')] -[2023-10-15 18:07:40,720][52833] Updated weights for policy 0, policy_version 83720 (0.0010) -[2023-10-15 18:07:41,086][52833] Updated weights for policy 0, policy_version 83730 (0.0011) -[2023-10-15 18:07:41,377][52866] Updated weights for policy 1, policy_version 83970 (0.0007) -[2023-10-15 18:07:41,450][52833] Updated weights for policy 0, policy_version 83740 (0.0009) -[2023-10-15 18:07:41,774][52866] Updated weights for policy 1, policy_version 83980 (0.0009) -[2023-10-15 18:07:42,142][52866] Updated weights for policy 1, policy_version 83990 (0.0009) -[2023-10-15 18:07:42,504][52866] Updated weights for policy 1, policy_version 84000 (0.0009) -[2023-10-15 18:07:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 171769856. Throughput: 0: 1804.3, 1: 1804.6. Samples: 42950288. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:07:43,441][51532] Avg episode reward: [(0, '70.050'), (1, '56.230')] -[2023-10-15 18:07:43,449][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000084000_86016000.pth... -[2023-10-15 18:07:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000083744_85753856.pth... -[2023-10-15 18:07:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000082304_84279296.pth -[2023-10-15 18:07:43,488][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000082048_84017152.pth -[2023-10-15 18:07:43,493][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000084000_86016000.pth -[2023-10-15 18:07:43,494][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000083744_85753856.pth -[2023-10-15 18:07:45,256][52833] Updated weights for policy 0, policy_version 83750 (0.0008) -[2023-10-15 18:07:45,631][52833] Updated weights for policy 0, policy_version 83760 (0.0010) -[2023-10-15 18:07:45,998][52833] Updated weights for policy 0, policy_version 83770 (0.0008) -[2023-10-15 18:07:46,282][52866] Updated weights for policy 1, policy_version 84010 (0.0008) -[2023-10-15 18:07:46,644][52866] Updated weights for policy 1, policy_version 84020 (0.0010) -[2023-10-15 18:07:47,012][52866] Updated weights for policy 1, policy_version 84030 (0.0009) -[2023-10-15 18:07:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171835392. Throughput: 0: 1813.2, 1: 1816.3. Samples: 42961994. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:07:48,442][51532] Avg episode reward: [(0, '72.080'), (1, '58.020')] -[2023-10-15 18:07:49,651][52833] Updated weights for policy 0, policy_version 83780 (0.0010) -[2023-10-15 18:07:50,014][52833] Updated weights for policy 0, policy_version 83790 (0.0007) -[2023-10-15 18:07:50,384][52833] Updated weights for policy 0, policy_version 83800 (0.0009) -[2023-10-15 18:07:50,936][52866] Updated weights for policy 1, policy_version 84040 (0.0009) -[2023-10-15 18:07:51,300][52866] Updated weights for policy 1, policy_version 84050 (0.0009) -[2023-10-15 18:07:51,663][52866] Updated weights for policy 1, policy_version 84060 (0.0010) -[2023-10-15 18:07:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171900928. Throughput: 0: 1800.9, 1: 1804.0. Samples: 42982636. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:07:53,441][51532] Avg episode reward: [(0, '75.380'), (1, '56.810')] -[2023-10-15 18:07:54,137][52833] Updated weights for policy 0, policy_version 83810 (0.0008) -[2023-10-15 18:07:54,515][52833] Updated weights for policy 0, policy_version 83820 (0.0010) -[2023-10-15 18:07:54,878][52833] Updated weights for policy 0, policy_version 83830 (0.0011) -[2023-10-15 18:07:55,249][52833] Updated weights for policy 0, policy_version 83840 (0.0009) -[2023-10-15 18:07:55,402][52866] Updated weights for policy 1, policy_version 84070 (0.0010) -[2023-10-15 18:07:55,761][52866] Updated weights for policy 1, policy_version 84080 (0.0007) -[2023-10-15 18:07:56,120][52866] Updated weights for policy 1, policy_version 84090 (0.0007) -[2023-10-15 18:07:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171966464. Throughput: 0: 1809.8, 1: 1795.1. Samples: 43005288. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:07:58,441][51532] Avg episode reward: [(0, '75.220'), (1, '57.330')] -[2023-10-15 18:07:58,836][52833] Updated weights for policy 0, policy_version 83850 (0.0008) -[2023-10-15 18:07:59,205][52833] Updated weights for policy 0, policy_version 83860 (0.0009) -[2023-10-15 18:07:59,583][52833] Updated weights for policy 0, policy_version 83870 (0.0009) -[2023-10-15 18:07:59,993][52866] Updated weights for policy 1, policy_version 84100 (0.0008) -[2023-10-15 18:08:00,352][52866] Updated weights for policy 1, policy_version 84110 (0.0009) -[2023-10-15 18:08:00,718][52866] Updated weights for policy 1, policy_version 84120 (0.0008) -[2023-10-15 18:08:03,213][52833] Updated weights for policy 0, policy_version 83880 (0.0009) -[2023-10-15 18:08:03,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 172032000. Throughput: 0: 1809.5, 1: 1797.4. Samples: 43015398. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:08:03,442][51532] Avg episode reward: [(0, '74.890'), (1, '59.490')] -[2023-10-15 18:08:03,582][52833] Updated weights for policy 0, policy_version 83890 (0.0009) -[2023-10-15 18:08:03,949][52833] Updated weights for policy 0, policy_version 83900 (0.0008) -[2023-10-15 18:08:04,484][52866] Updated weights for policy 1, policy_version 84130 (0.0007) -[2023-10-15 18:08:04,842][52866] Updated weights for policy 1, policy_version 84140 (0.0009) -[2023-10-15 18:08:05,220][52866] Updated weights for policy 1, policy_version 84150 (0.0010) -[2023-10-15 18:08:05,576][52866] Updated weights for policy 1, policy_version 84160 (0.0011) -[2023-10-15 18:08:07,746][52833] Updated weights for policy 0, policy_version 83910 (0.0008) -[2023-10-15 18:08:08,127][52833] Updated weights for policy 0, policy_version 83920 (0.0007) -[2023-10-15 18:08:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 172097536. Throughput: 0: 1804.4, 1: 1784.3. Samples: 43037348. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:08:08,441][51532] Avg episode reward: [(0, '75.590'), (1, '60.220')] -[2023-10-15 18:08:08,503][52833] Updated weights for policy 0, policy_version 83930 (0.0009) -[2023-10-15 18:08:09,328][52866] Updated weights for policy 1, policy_version 84170 (0.0008) -[2023-10-15 18:08:09,702][52866] Updated weights for policy 1, policy_version 84180 (0.0007) -[2023-10-15 18:08:10,067][52866] Updated weights for policy 1, policy_version 84190 (0.0008) -[2023-10-15 18:08:12,410][52833] Updated weights for policy 0, policy_version 83940 (0.0008) -[2023-10-15 18:08:12,787][52833] Updated weights for policy 0, policy_version 83950 (0.0008) -[2023-10-15 18:08:13,152][52833] Updated weights for policy 0, policy_version 83960 (0.0010) -[2023-10-15 18:08:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 172163072. Throughput: 0: 1814.3, 1: 1797.0. Samples: 43059220. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-15 18:08:13,441][51532] Avg episode reward: [(0, '75.300'), (1, '59.660')] -[2023-10-15 18:08:13,711][52866] Updated weights for policy 1, policy_version 84200 (0.0009) -[2023-10-15 18:08:14,086][52866] Updated weights for policy 1, policy_version 84210 (0.0009) -[2023-10-15 18:08:14,442][52866] Updated weights for policy 1, policy_version 84220 (0.0009) -[2023-10-15 18:08:16,735][52833] Updated weights for policy 0, policy_version 83970 (0.0010) -[2023-10-15 18:08:17,106][52833] Updated weights for policy 0, policy_version 83980 (0.0008) -[2023-10-15 18:08:17,474][52833] Updated weights for policy 0, policy_version 83990 (0.0007) -[2023-10-15 18:08:17,844][52833] Updated weights for policy 0, policy_version 84000 (0.0009) -[2023-10-15 18:08:18,292][52866] Updated weights for policy 1, policy_version 84230 (0.0008) -[2023-10-15 18:08:18,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 172261376. Throughput: 0: 1801.5, 1: 1785.3. Samples: 43069802. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:18,441][51532] Avg episode reward: [(0, '75.500'), (1, '60.920')] -[2023-10-15 18:08:18,655][52866] Updated weights for policy 1, policy_version 84240 (0.0008) -[2023-10-15 18:08:19,025][52866] Updated weights for policy 1, policy_version 84250 (0.0007) -[2023-10-15 18:08:21,630][52833] Updated weights for policy 0, policy_version 84010 (0.0007) -[2023-10-15 18:08:22,002][52833] Updated weights for policy 0, policy_version 84020 (0.0009) -[2023-10-15 18:08:22,375][52833] Updated weights for policy 0, policy_version 84030 (0.0008) -[2023-10-15 18:08:22,731][52866] Updated weights for policy 1, policy_version 84260 (0.0009) -[2023-10-15 18:08:23,099][52866] Updated weights for policy 1, policy_version 84270 (0.0009) -[2023-10-15 18:08:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 172326912. Throughput: 0: 1813.5, 1: 1796.9. Samples: 43091662. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:23,441][51532] Avg episode reward: [(0, '77.440'), (1, '60.010')] -[2023-10-15 18:08:23,465][52866] Updated weights for policy 1, policy_version 84280 (0.0009) -[2023-10-15 18:08:25,947][52833] Updated weights for policy 0, policy_version 84040 (0.0009) -[2023-10-15 18:08:26,310][52833] Updated weights for policy 0, policy_version 84050 (0.0008) -[2023-10-15 18:08:26,680][52833] Updated weights for policy 0, policy_version 84060 (0.0009) -[2023-10-15 18:08:27,362][52866] Updated weights for policy 1, policy_version 84290 (0.0007) -[2023-10-15 18:08:27,760][52866] Updated weights for policy 1, policy_version 84300 (0.0008) -[2023-10-15 18:08:28,129][52866] Updated weights for policy 1, policy_version 84310 (0.0009) -[2023-10-15 18:08:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 172392448. Throughput: 0: 1810.1, 1: 1798.8. Samples: 43112692. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:28,442][51532] Avg episode reward: [(0, '76.950'), (1, '60.350')] -[2023-10-15 18:08:28,490][52866] Updated weights for policy 1, policy_version 84320 (0.0008) -[2023-10-15 18:08:30,395][52833] Updated weights for policy 0, policy_version 84070 (0.0007) -[2023-10-15 18:08:30,769][52833] Updated weights for policy 0, policy_version 84080 (0.0008) -[2023-10-15 18:08:31,132][52833] Updated weights for policy 0, policy_version 84090 (0.0007) -[2023-10-15 18:08:32,150][52866] Updated weights for policy 1, policy_version 84330 (0.0010) -[2023-10-15 18:08:32,521][52866] Updated weights for policy 1, policy_version 84340 (0.0007) -[2023-10-15 18:08:32,888][52866] Updated weights for policy 1, policy_version 84350 (0.0007) -[2023-10-15 18:08:33,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172490752. Throughput: 0: 1813.2, 1: 1785.4. Samples: 43123932. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:33,441][51532] Avg episode reward: [(0, '79.180'), (1, '62.760')] -[2023-10-15 18:08:34,819][52833] Updated weights for policy 0, policy_version 84100 (0.0009) -[2023-10-15 18:08:35,195][52833] Updated weights for policy 0, policy_version 84110 (0.0007) -[2023-10-15 18:08:35,561][52833] Updated weights for policy 0, policy_version 84120 (0.0007) -[2023-10-15 18:08:36,616][52866] Updated weights for policy 1, policy_version 84360 (0.0008) -[2023-10-15 18:08:36,992][52866] Updated weights for policy 1, policy_version 84370 (0.0008) -[2023-10-15 18:08:37,358][52866] Updated weights for policy 1, policy_version 84380 (0.0007) -[2023-10-15 18:08:38,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 172556288. Throughput: 0: 1807.7, 1: 1804.3. Samples: 43145176. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:38,441][51532] Avg episode reward: [(0, '79.820'), (1, '59.790')] -[2023-10-15 18:08:39,214][52833] Updated weights for policy 0, policy_version 84130 (0.0007) -[2023-10-15 18:08:39,577][52833] Updated weights for policy 0, policy_version 84140 (0.0007) -[2023-10-15 18:08:39,953][52833] Updated weights for policy 0, policy_version 84150 (0.0007) -[2023-10-15 18:08:40,322][52833] Updated weights for policy 0, policy_version 84160 (0.0010) -[2023-10-15 18:08:41,071][52866] Updated weights for policy 1, policy_version 84390 (0.0009) -[2023-10-15 18:08:41,437][52866] Updated weights for policy 1, policy_version 84400 (0.0007) -[2023-10-15 18:08:41,799][52866] Updated weights for policy 1, policy_version 84410 (0.0007) -[2023-10-15 18:08:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172621824. Throughput: 0: 1797.0, 1: 1791.9. Samples: 43166788. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:43,441][51532] Avg episode reward: [(0, '76.240'), (1, '63.100')] -[2023-10-15 18:08:44,183][52833] Updated weights for policy 0, policy_version 84170 (0.0009) -[2023-10-15 18:08:44,544][52833] Updated weights for policy 0, policy_version 84180 (0.0007) -[2023-10-15 18:08:44,910][52833] Updated weights for policy 0, policy_version 84190 (0.0007) -[2023-10-15 18:08:45,479][52866] Updated weights for policy 1, policy_version 84420 (0.0010) -[2023-10-15 18:08:45,848][52866] Updated weights for policy 1, policy_version 84430 (0.0009) -[2023-10-15 18:08:46,213][52866] Updated weights for policy 1, policy_version 84440 (0.0008) -[2023-10-15 18:08:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172687360. Throughput: 0: 1794.8, 1: 1807.1. Samples: 43177484. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:48,441][51532] Avg episode reward: [(0, '72.530'), (1, '64.260')] -[2023-10-15 18:08:48,697][52833] Updated weights for policy 0, policy_version 84200 (0.0008) -[2023-10-15 18:08:49,058][52833] Updated weights for policy 0, policy_version 84210 (0.0008) -[2023-10-15 18:08:49,437][52833] Updated weights for policy 0, policy_version 84220 (0.0008) -[2023-10-15 18:08:49,857][52866] Updated weights for policy 1, policy_version 84450 (0.0010) -[2023-10-15 18:08:50,221][52866] Updated weights for policy 1, policy_version 84460 (0.0008) -[2023-10-15 18:08:50,599][52866] Updated weights for policy 1, policy_version 84470 (0.0007) -[2023-10-15 18:08:50,964][52866] Updated weights for policy 1, policy_version 84480 (0.0009) -[2023-10-15 18:08:53,118][52833] Updated weights for policy 0, policy_version 84230 (0.0008) -[2023-10-15 18:08:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172752896. Throughput: 0: 1801.0, 1: 1798.8. Samples: 43199340. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:53,441][51532] Avg episode reward: [(0, '72.080'), (1, '64.360')] -[2023-10-15 18:08:53,490][52833] Updated weights for policy 0, policy_version 84240 (0.0009) -[2023-10-15 18:08:53,862][52833] Updated weights for policy 0, policy_version 84250 (0.0008) -[2023-10-15 18:08:54,608][52866] Updated weights for policy 1, policy_version 84490 (0.0009) -[2023-10-15 18:08:54,976][52866] Updated weights for policy 1, policy_version 84500 (0.0009) -[2023-10-15 18:08:55,348][52866] Updated weights for policy 1, policy_version 84510 (0.0008) -[2023-10-15 18:08:57,491][52833] Updated weights for policy 0, policy_version 84260 (0.0008) -[2023-10-15 18:08:57,855][52833] Updated weights for policy 0, policy_version 84270 (0.0010) -[2023-10-15 18:08:58,228][52833] Updated weights for policy 0, policy_version 84280 (0.0007) -[2023-10-15 18:08:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 172818432. Throughput: 0: 1808.3, 1: 1796.3. Samples: 43221426. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:08:58,442][51532] Avg episode reward: [(0, '70.380'), (1, '64.920')] -[2023-10-15 18:08:59,163][52866] Updated weights for policy 1, policy_version 84520 (0.0010) -[2023-10-15 18:08:59,521][52866] Updated weights for policy 1, policy_version 84530 (0.0010) -[2023-10-15 18:08:59,894][52866] Updated weights for policy 1, policy_version 84540 (0.0010) -[2023-10-15 18:09:01,943][52833] Updated weights for policy 0, policy_version 84290 (0.0009) -[2023-10-15 18:09:02,310][52833] Updated weights for policy 0, policy_version 84300 (0.0010) -[2023-10-15 18:09:02,670][52833] Updated weights for policy 0, policy_version 84310 (0.0010) -[2023-10-15 18:09:03,037][52833] Updated weights for policy 0, policy_version 84320 (0.0007) -[2023-10-15 18:09:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 172916736. Throughput: 0: 1802.6, 1: 1799.4. Samples: 43231890. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-15 18:09:03,441][51532] Avg episode reward: [(0, '71.750'), (1, '64.290')] -[2023-10-15 18:09:03,600][52866] Updated weights for policy 1, policy_version 84550 (0.0008) -[2023-10-15 18:09:03,966][52866] Updated weights for policy 1, policy_version 84560 (0.0009) -[2023-10-15 18:09:04,339][52866] Updated weights for policy 1, policy_version 84570 (0.0011) -[2023-10-15 18:09:06,873][52833] Updated weights for policy 0, policy_version 84330 (0.0009) -[2023-10-15 18:09:07,247][52833] Updated weights for policy 0, policy_version 84340 (0.0007) -[2023-10-15 18:09:07,613][52833] Updated weights for policy 0, policy_version 84350 (0.0008) -[2023-10-15 18:09:08,133][52866] Updated weights for policy 1, policy_version 84580 (0.0008) -[2023-10-15 18:09:08,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 172982272. Throughput: 0: 1805.2, 1: 1801.7. Samples: 43253972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:08,441][51532] Avg episode reward: [(0, '71.190'), (1, '61.690')] -[2023-10-15 18:09:08,501][52866] Updated weights for policy 1, policy_version 84590 (0.0010) -[2023-10-15 18:09:08,870][52866] Updated weights for policy 1, policy_version 84600 (0.0009) -[2023-10-15 18:09:11,403][52833] Updated weights for policy 0, policy_version 84360 (0.0011) -[2023-10-15 18:09:11,780][52833] Updated weights for policy 0, policy_version 84370 (0.0008) -[2023-10-15 18:09:12,142][52833] Updated weights for policy 0, policy_version 84380 (0.0007) -[2023-10-15 18:09:12,589][52866] Updated weights for policy 1, policy_version 84610 (0.0007) -[2023-10-15 18:09:12,978][52866] Updated weights for policy 1, policy_version 84620 (0.0008) -[2023-10-15 18:09:13,330][52866] Updated weights for policy 1, policy_version 84630 (0.0007) -[2023-10-15 18:09:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 173047808. Throughput: 0: 1795.0, 1: 1814.5. Samples: 43275118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:13,442][51532] Avg episode reward: [(0, '71.150'), (1, '60.590')] -[2023-10-15 18:09:13,694][52866] Updated weights for policy 1, policy_version 84640 (0.0008) -[2023-10-15 18:09:15,800][52833] Updated weights for policy 0, policy_version 84390 (0.0009) -[2023-10-15 18:09:16,159][52833] Updated weights for policy 0, policy_version 84400 (0.0012) -[2023-10-15 18:09:16,529][52833] Updated weights for policy 0, policy_version 84410 (0.0008) -[2023-10-15 18:09:17,398][52866] Updated weights for policy 1, policy_version 84650 (0.0011) -[2023-10-15 18:09:17,760][52866] Updated weights for policy 1, policy_version 84660 (0.0007) -[2023-10-15 18:09:18,128][52866] Updated weights for policy 1, policy_version 84670 (0.0007) -[2023-10-15 18:09:18,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 173146112. Throughput: 0: 1809.6, 1: 1807.7. Samples: 43286710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:18,441][51532] Avg episode reward: [(0, '72.860'), (1, '62.710')] -[2023-10-15 18:09:20,354][52833] Updated weights for policy 0, policy_version 84420 (0.0007) -[2023-10-15 18:09:20,722][52833] Updated weights for policy 0, policy_version 84430 (0.0007) -[2023-10-15 18:09:21,086][52833] Updated weights for policy 0, policy_version 84440 (0.0008) -[2023-10-15 18:09:21,837][52866] Updated weights for policy 1, policy_version 84680 (0.0010) -[2023-10-15 18:09:22,202][52866] Updated weights for policy 1, policy_version 84690 (0.0008) -[2023-10-15 18:09:22,567][52866] Updated weights for policy 1, policy_version 84700 (0.0009) -[2023-10-15 18:09:23,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 173211648. Throughput: 0: 1796.5, 1: 1808.9. Samples: 43307420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:23,441][51532] Avg episode reward: [(0, '72.070'), (1, '60.700')] -[2023-10-15 18:09:24,658][52833] Updated weights for policy 0, policy_version 84450 (0.0010) -[2023-10-15 18:09:25,022][52833] Updated weights for policy 0, policy_version 84460 (0.0010) -[2023-10-15 18:09:25,385][52833] Updated weights for policy 0, policy_version 84470 (0.0009) -[2023-10-15 18:09:25,756][52833] Updated weights for policy 0, policy_version 84480 (0.0007) -[2023-10-15 18:09:26,353][52866] Updated weights for policy 1, policy_version 84710 (0.0011) -[2023-10-15 18:09:26,718][52866] Updated weights for policy 1, policy_version 84720 (0.0010) -[2023-10-15 18:09:27,099][52866] Updated weights for policy 1, policy_version 84730 (0.0010) -[2023-10-15 18:09:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 173277184. Throughput: 0: 1803.1, 1: 1800.4. Samples: 43328948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:28,441][51532] Avg episode reward: [(0, '66.830'), (1, '58.950')] -[2023-10-15 18:09:29,471][52833] Updated weights for policy 0, policy_version 84490 (0.0009) -[2023-10-15 18:09:29,836][52833] Updated weights for policy 0, policy_version 84500 (0.0012) -[2023-10-15 18:09:30,212][52833] Updated weights for policy 0, policy_version 84510 (0.0009) -[2023-10-15 18:09:30,905][52866] Updated weights for policy 1, policy_version 84740 (0.0007) -[2023-10-15 18:09:31,265][52866] Updated weights for policy 1, policy_version 84750 (0.0009) -[2023-10-15 18:09:31,633][52866] Updated weights for policy 1, policy_version 84760 (0.0010) -[2023-10-15 18:09:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 173342720. Throughput: 0: 1803.0, 1: 1808.2. Samples: 43339988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:33,442][51532] Avg episode reward: [(0, '67.940'), (1, '59.640')] -[2023-10-15 18:09:33,902][52833] Updated weights for policy 0, policy_version 84520 (0.0010) -[2023-10-15 18:09:34,271][52833] Updated weights for policy 0, policy_version 84530 (0.0009) -[2023-10-15 18:09:34,641][52833] Updated weights for policy 0, policy_version 84540 (0.0008) -[2023-10-15 18:09:35,391][52866] Updated weights for policy 1, policy_version 84770 (0.0009) -[2023-10-15 18:09:35,753][52866] Updated weights for policy 1, policy_version 84780 (0.0008) -[2023-10-15 18:09:36,108][52866] Updated weights for policy 1, policy_version 84790 (0.0009) -[2023-10-15 18:09:36,474][52866] Updated weights for policy 1, policy_version 84800 (0.0007) -[2023-10-15 18:09:38,425][52833] Updated weights for policy 0, policy_version 84550 (0.0009) -[2023-10-15 18:09:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 173408256. Throughput: 0: 1801.9, 1: 1797.0. Samples: 43361288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:38,441][51532] Avg episode reward: [(0, '68.890'), (1, '62.250')] -[2023-10-15 18:09:38,789][52833] Updated weights for policy 0, policy_version 84560 (0.0010) -[2023-10-15 18:09:39,161][52833] Updated weights for policy 0, policy_version 84570 (0.0010) -[2023-10-15 18:09:40,183][52866] Updated weights for policy 1, policy_version 84810 (0.0008) -[2023-10-15 18:09:40,549][52866] Updated weights for policy 1, policy_version 84820 (0.0007) -[2023-10-15 18:09:40,908][52866] Updated weights for policy 1, policy_version 84830 (0.0010) -[2023-10-15 18:09:42,946][52833] Updated weights for policy 0, policy_version 84580 (0.0010) -[2023-10-15 18:09:43,322][52833] Updated weights for policy 0, policy_version 84590 (0.0010) -[2023-10-15 18:09:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 173473792. Throughput: 0: 1815.3, 1: 1792.4. Samples: 43383776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:43,442][51532] Avg episode reward: [(0, '70.420'), (1, '62.730')] -[2023-10-15 18:09:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000084832_86867968.pth... -[2023-10-15 18:09:43,487][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000083168_85164032.pth -[2023-10-15 18:09:43,684][52833] Updated weights for policy 0, policy_version 84600 (0.0009) -[2023-10-15 18:09:43,984][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000084608_86638592.pth... -[2023-10-15 18:09:44,021][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000082912_84901888.pth -[2023-10-15 18:09:44,847][52866] Updated weights for policy 1, policy_version 84840 (0.0010) -[2023-10-15 18:09:45,215][52866] Updated weights for policy 1, policy_version 84850 (0.0009) -[2023-10-15 18:09:45,582][52866] Updated weights for policy 1, policy_version 84860 (0.0009) -[2023-10-15 18:09:47,456][52833] Updated weights for policy 0, policy_version 84610 (0.0009) -[2023-10-15 18:09:47,827][52833] Updated weights for policy 0, policy_version 84620 (0.0009) -[2023-10-15 18:09:48,198][52833] Updated weights for policy 0, policy_version 84630 (0.0009) -[2023-10-15 18:09:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 173539328. Throughput: 0: 1803.1, 1: 1790.2. Samples: 43393590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:48,441][51532] Avg episode reward: [(0, '70.540'), (1, '62.380')] -[2023-10-15 18:09:48,568][52833] Updated weights for policy 0, policy_version 84640 (0.0008) -[2023-10-15 18:09:49,361][52866] Updated weights for policy 1, policy_version 84870 (0.0008) -[2023-10-15 18:09:49,720][52866] Updated weights for policy 1, policy_version 84880 (0.0008) -[2023-10-15 18:09:50,089][52866] Updated weights for policy 1, policy_version 84890 (0.0008) -[2023-10-15 18:09:52,490][52833] Updated weights for policy 0, policy_version 84650 (0.0008) -[2023-10-15 18:09:52,856][52833] Updated weights for policy 0, policy_version 84660 (0.0011) -[2023-10-15 18:09:53,227][52833] Updated weights for policy 0, policy_version 84670 (0.0010) -[2023-10-15 18:09:53,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 173637632. Throughput: 0: 1816.9, 1: 1785.2. Samples: 43416064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:53,442][51532] Avg episode reward: [(0, '72.460'), (1, '62.340')] -[2023-10-15 18:09:53,754][52866] Updated weights for policy 1, policy_version 84900 (0.0009) -[2023-10-15 18:09:54,127][52866] Updated weights for policy 1, policy_version 84910 (0.0011) -[2023-10-15 18:09:54,491][52866] Updated weights for policy 1, policy_version 84920 (0.0008) -[2023-10-15 18:09:56,863][52833] Updated weights for policy 0, policy_version 84680 (0.0009) -[2023-10-15 18:09:57,242][52833] Updated weights for policy 0, policy_version 84690 (0.0008) -[2023-10-15 18:09:57,622][52833] Updated weights for policy 0, policy_version 84700 (0.0007) -[2023-10-15 18:09:58,332][52866] Updated weights for policy 1, policy_version 84930 (0.0009) -[2023-10-15 18:09:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 173703168. Throughput: 0: 1798.8, 1: 1795.2. Samples: 43436848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:09:58,441][51532] Avg episode reward: [(0, '73.950'), (1, '60.700')] -[2023-10-15 18:09:58,741][52866] Updated weights for policy 1, policy_version 84940 (0.0011) -[2023-10-15 18:09:59,105][52866] Updated weights for policy 1, policy_version 84950 (0.0007) -[2023-10-15 18:09:59,466][52866] Updated weights for policy 1, policy_version 84960 (0.0007) -[2023-10-15 18:10:01,404][52833] Updated weights for policy 0, policy_version 84710 (0.0007) -[2023-10-15 18:10:01,770][52833] Updated weights for policy 0, policy_version 84720 (0.0010) -[2023-10-15 18:10:02,137][52833] Updated weights for policy 0, policy_version 84730 (0.0009) -[2023-10-15 18:10:03,113][52866] Updated weights for policy 1, policy_version 84970 (0.0008) -[2023-10-15 18:10:03,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 173768704. Throughput: 0: 1802.4, 1: 1780.3. Samples: 43447936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:03,442][51532] Avg episode reward: [(0, '73.660'), (1, '63.880')] -[2023-10-15 18:10:03,476][52866] Updated weights for policy 1, policy_version 84980 (0.0008) -[2023-10-15 18:10:03,844][52866] Updated weights for policy 1, policy_version 84990 (0.0011) -[2023-10-15 18:10:05,800][52833] Updated weights for policy 0, policy_version 84740 (0.0008) -[2023-10-15 18:10:06,165][52833] Updated weights for policy 0, policy_version 84750 (0.0008) -[2023-10-15 18:10:06,539][52833] Updated weights for policy 0, policy_version 84760 (0.0008) -[2023-10-15 18:10:07,498][52866] Updated weights for policy 1, policy_version 85000 (0.0008) -[2023-10-15 18:10:07,864][52866] Updated weights for policy 1, policy_version 85010 (0.0009) -[2023-10-15 18:10:08,226][52866] Updated weights for policy 1, policy_version 85020 (0.0008) -[2023-10-15 18:10:08,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 173867008. Throughput: 0: 1797.0, 1: 1800.3. Samples: 43469296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:08,442][51532] Avg episode reward: [(0, '72.420'), (1, '65.120')] -[2023-10-15 18:10:10,222][52833] Updated weights for policy 0, policy_version 84770 (0.0008) -[2023-10-15 18:10:10,594][52833] Updated weights for policy 0, policy_version 84780 (0.0010) -[2023-10-15 18:10:10,954][52833] Updated weights for policy 0, policy_version 84790 (0.0009) -[2023-10-15 18:10:11,326][52833] Updated weights for policy 0, policy_version 84800 (0.0009) -[2023-10-15 18:10:11,909][52866] Updated weights for policy 1, policy_version 85030 (0.0008) -[2023-10-15 18:10:12,270][52866] Updated weights for policy 1, policy_version 85040 (0.0010) -[2023-10-15 18:10:12,634][52866] Updated weights for policy 1, policy_version 85050 (0.0007) -[2023-10-15 18:10:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 173932544. Throughput: 0: 1798.0, 1: 1796.6. Samples: 43490704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:13,442][51532] Avg episode reward: [(0, '70.660'), (1, '63.480')] -[2023-10-15 18:10:15,047][52833] Updated weights for policy 0, policy_version 84810 (0.0009) -[2023-10-15 18:10:15,409][52833] Updated weights for policy 0, policy_version 84820 (0.0008) -[2023-10-15 18:10:15,775][52833] Updated weights for policy 0, policy_version 84830 (0.0009) -[2023-10-15 18:10:16,188][52866] Updated weights for policy 1, policy_version 85060 (0.0008) -[2023-10-15 18:10:16,555][52866] Updated weights for policy 1, policy_version 85070 (0.0007) -[2023-10-15 18:10:16,927][52866] Updated weights for policy 1, policy_version 85080 (0.0008) -[2023-10-15 18:10:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 173998080. Throughput: 0: 1795.2, 1: 1808.6. Samples: 43502158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:18,442][51532] Avg episode reward: [(0, '69.450'), (1, '63.690')] -[2023-10-15 18:10:19,622][52833] Updated weights for policy 0, policy_version 84840 (0.0007) -[2023-10-15 18:10:19,994][52833] Updated weights for policy 0, policy_version 84850 (0.0007) -[2023-10-15 18:10:20,358][52833] Updated weights for policy 0, policy_version 84860 (0.0009) -[2023-10-15 18:10:20,849][52866] Updated weights for policy 1, policy_version 85090 (0.0007) -[2023-10-15 18:10:21,223][52866] Updated weights for policy 1, policy_version 85100 (0.0008) -[2023-10-15 18:10:21,596][52866] Updated weights for policy 1, policy_version 85110 (0.0008) -[2023-10-15 18:10:21,959][52866] Updated weights for policy 1, policy_version 85120 (0.0009) -[2023-10-15 18:10:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 174063616. Throughput: 0: 1792.4, 1: 1805.2. Samples: 43523182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:23,442][51532] Avg episode reward: [(0, '70.800'), (1, '62.730')] -[2023-10-15 18:10:23,918][52833] Updated weights for policy 0, policy_version 84870 (0.0008) -[2023-10-15 18:10:24,283][52833] Updated weights for policy 0, policy_version 84880 (0.0007) -[2023-10-15 18:10:24,662][52833] Updated weights for policy 0, policy_version 84890 (0.0008) -[2023-10-15 18:10:25,768][52866] Updated weights for policy 1, policy_version 85130 (0.0010) -[2023-10-15 18:10:26,136][52866] Updated weights for policy 1, policy_version 85140 (0.0011) -[2023-10-15 18:10:26,510][52866] Updated weights for policy 1, policy_version 85150 (0.0008) -[2023-10-15 18:10:28,339][52833] Updated weights for policy 0, policy_version 84900 (0.0010) -[2023-10-15 18:10:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 174129152. Throughput: 0: 1802.9, 1: 1802.4. Samples: 43546014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:28,442][51532] Avg episode reward: [(0, '71.740'), (1, '63.550')] -[2023-10-15 18:10:28,707][52833] Updated weights for policy 0, policy_version 84910 (0.0011) -[2023-10-15 18:10:29,069][52833] Updated weights for policy 0, policy_version 84920 (0.0010) -[2023-10-15 18:10:30,249][52866] Updated weights for policy 1, policy_version 85160 (0.0008) -[2023-10-15 18:10:30,619][52866] Updated weights for policy 1, policy_version 85170 (0.0007) -[2023-10-15 18:10:30,997][52866] Updated weights for policy 1, policy_version 85180 (0.0008) -[2023-10-15 18:10:32,873][52833] Updated weights for policy 0, policy_version 84930 (0.0009) -[2023-10-15 18:10:33,245][52833] Updated weights for policy 0, policy_version 84940 (0.0008) -[2023-10-15 18:10:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 174194688. Throughput: 0: 1797.9, 1: 1813.8. Samples: 43556116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:33,441][51532] Avg episode reward: [(0, '72.900'), (1, '63.560')] -[2023-10-15 18:10:33,607][52833] Updated weights for policy 0, policy_version 84950 (0.0010) -[2023-10-15 18:10:33,978][52833] Updated weights for policy 0, policy_version 84960 (0.0009) -[2023-10-15 18:10:34,848][52866] Updated weights for policy 1, policy_version 85190 (0.0010) -[2023-10-15 18:10:35,209][52866] Updated weights for policy 1, policy_version 85200 (0.0008) -[2023-10-15 18:10:35,572][52866] Updated weights for policy 1, policy_version 85210 (0.0010) -[2023-10-15 18:10:37,788][52833] Updated weights for policy 0, policy_version 84970 (0.0011) -[2023-10-15 18:10:38,151][52833] Updated weights for policy 0, policy_version 84980 (0.0008) -[2023-10-15 18:10:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174260224. Throughput: 0: 1801.2, 1: 1805.0. Samples: 43578342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:38,442][51532] Avg episode reward: [(0, '70.730'), (1, '64.490')] -[2023-10-15 18:10:38,530][52833] Updated weights for policy 0, policy_version 84990 (0.0007) -[2023-10-15 18:10:39,314][52866] Updated weights for policy 1, policy_version 85220 (0.0007) -[2023-10-15 18:10:39,685][52866] Updated weights for policy 1, policy_version 85230 (0.0008) -[2023-10-15 18:10:40,053][52866] Updated weights for policy 1, policy_version 85240 (0.0008) -[2023-10-15 18:10:42,298][52833] Updated weights for policy 0, policy_version 85000 (0.0011) -[2023-10-15 18:10:42,679][52833] Updated weights for policy 0, policy_version 85010 (0.0009) -[2023-10-15 18:10:43,043][52833] Updated weights for policy 0, policy_version 85020 (0.0008) -[2023-10-15 18:10:43,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 174358528. Throughput: 0: 1810.6, 1: 1808.5. Samples: 43599710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:43,442][51532] Avg episode reward: [(0, '71.140'), (1, '62.480')] -[2023-10-15 18:10:43,681][52866] Updated weights for policy 1, policy_version 85250 (0.0010) -[2023-10-15 18:10:44,063][52866] Updated weights for policy 1, policy_version 85260 (0.0009) -[2023-10-15 18:10:44,430][52866] Updated weights for policy 1, policy_version 85270 (0.0008) -[2023-10-15 18:10:44,798][52866] Updated weights for policy 1, policy_version 85280 (0.0008) -[2023-10-15 18:10:46,865][52833] Updated weights for policy 0, policy_version 85030 (0.0009) -[2023-10-15 18:10:47,228][52833] Updated weights for policy 0, policy_version 85040 (0.0008) -[2023-10-15 18:10:47,597][52833] Updated weights for policy 0, policy_version 85050 (0.0011) -[2023-10-15 18:10:48,408][52866] Updated weights for policy 1, policy_version 85290 (0.0009) -[2023-10-15 18:10:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 174424064. Throughput: 0: 1802.7, 1: 1810.3. Samples: 43610520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:48,441][51532] Avg episode reward: [(0, '68.640'), (1, '61.380')] -[2023-10-15 18:10:48,780][52866] Updated weights for policy 1, policy_version 85300 (0.0009) -[2023-10-15 18:10:49,143][52866] Updated weights for policy 1, policy_version 85310 (0.0008) -[2023-10-15 18:10:51,212][52833] Updated weights for policy 0, policy_version 85060 (0.0009) -[2023-10-15 18:10:51,584][52833] Updated weights for policy 0, policy_version 85070 (0.0008) -[2023-10-15 18:10:51,962][52833] Updated weights for policy 0, policy_version 85080 (0.0007) -[2023-10-15 18:10:52,730][52866] Updated weights for policy 1, policy_version 85320 (0.0008) -[2023-10-15 18:10:53,097][52866] Updated weights for policy 1, policy_version 85330 (0.0007) -[2023-10-15 18:10:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 174489600. Throughput: 0: 1815.4, 1: 1807.3. Samples: 43632316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:10:53,441][51532] Avg episode reward: [(0, '68.830'), (1, '62.100')] -[2023-10-15 18:10:53,467][52866] Updated weights for policy 1, policy_version 85340 (0.0008) -[2023-10-15 18:10:55,534][52833] Updated weights for policy 0, policy_version 85090 (0.0007) -[2023-10-15 18:10:55,900][52833] Updated weights for policy 0, policy_version 85100 (0.0007) -[2023-10-15 18:10:56,281][52833] Updated weights for policy 0, policy_version 85110 (0.0009) -[2023-10-15 18:10:56,644][52833] Updated weights for policy 0, policy_version 85120 (0.0009) -[2023-10-15 18:10:57,163][52866] Updated weights for policy 1, policy_version 85350 (0.0009) -[2023-10-15 18:10:57,542][52866] Updated weights for policy 1, policy_version 85360 (0.0010) -[2023-10-15 18:10:57,898][52866] Updated weights for policy 1, policy_version 85370 (0.0009) -[2023-10-15 18:10:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 174587904. Throughput: 0: 1808.1, 1: 1809.1. Samples: 43653476. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:10:58,441][51532] Avg episode reward: [(0, '68.960'), (1, '61.910')] -[2023-10-15 18:11:00,389][52833] Updated weights for policy 0, policy_version 85130 (0.0008) -[2023-10-15 18:11:00,754][52833] Updated weights for policy 0, policy_version 85140 (0.0009) -[2023-10-15 18:11:01,123][52833] Updated weights for policy 0, policy_version 85150 (0.0008) -[2023-10-15 18:11:01,630][52866] Updated weights for policy 1, policy_version 85380 (0.0011) -[2023-10-15 18:11:01,991][52866] Updated weights for policy 1, policy_version 85390 (0.0011) -[2023-10-15 18:11:02,366][52866] Updated weights for policy 1, policy_version 85400 (0.0011) -[2023-10-15 18:11:03,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 174653440. Throughput: 0: 1821.4, 1: 1797.1. Samples: 43664988. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:03,442][51532] Avg episode reward: [(0, '68.910'), (1, '63.400')] -[2023-10-15 18:11:04,814][52833] Updated weights for policy 0, policy_version 85160 (0.0007) -[2023-10-15 18:11:05,186][52833] Updated weights for policy 0, policy_version 85170 (0.0011) -[2023-10-15 18:11:05,554][52833] Updated weights for policy 0, policy_version 85180 (0.0009) -[2023-10-15 18:11:06,292][52866] Updated weights for policy 1, policy_version 85410 (0.0011) -[2023-10-15 18:11:06,657][52866] Updated weights for policy 1, policy_version 85420 (0.0010) -[2023-10-15 18:11:07,020][52866] Updated weights for policy 1, policy_version 85430 (0.0011) -[2023-10-15 18:11:07,387][52866] Updated weights for policy 1, policy_version 85440 (0.0010) -[2023-10-15 18:11:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174718976. Throughput: 0: 1816.7, 1: 1807.0. Samples: 43686248. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:08,442][51532] Avg episode reward: [(0, '70.800'), (1, '62.550')] -[2023-10-15 18:11:09,222][52833] Updated weights for policy 0, policy_version 85190 (0.0010) -[2023-10-15 18:11:09,589][52833] Updated weights for policy 0, policy_version 85200 (0.0008) -[2023-10-15 18:11:09,959][52833] Updated weights for policy 0, policy_version 85210 (0.0007) -[2023-10-15 18:11:10,965][52866] Updated weights for policy 1, policy_version 85450 (0.0010) -[2023-10-15 18:11:11,334][52866] Updated weights for policy 1, policy_version 85460 (0.0010) -[2023-10-15 18:11:11,699][52866] Updated weights for policy 1, policy_version 85470 (0.0007) -[2023-10-15 18:11:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174784512. Throughput: 0: 1807.9, 1: 1806.8. Samples: 43708676. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:13,441][51532] Avg episode reward: [(0, '73.170'), (1, '63.530')] -[2023-10-15 18:11:13,717][52833] Updated weights for policy 0, policy_version 85220 (0.0007) -[2023-10-15 18:11:14,085][52833] Updated weights for policy 0, policy_version 85230 (0.0008) -[2023-10-15 18:11:14,462][52833] Updated weights for policy 0, policy_version 85240 (0.0007) -[2023-10-15 18:11:15,419][52866] Updated weights for policy 1, policy_version 85480 (0.0007) -[2023-10-15 18:11:15,787][52866] Updated weights for policy 1, policy_version 85490 (0.0010) -[2023-10-15 18:11:16,162][52866] Updated weights for policy 1, policy_version 85500 (0.0009) -[2023-10-15 18:11:18,203][52833] Updated weights for policy 0, policy_version 85250 (0.0008) -[2023-10-15 18:11:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174850048. Throughput: 0: 1810.4, 1: 1809.5. Samples: 43719012. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:18,442][51532] Avg episode reward: [(0, '72.050'), (1, '66.570')] -[2023-10-15 18:11:18,576][52833] Updated weights for policy 0, policy_version 85260 (0.0008) -[2023-10-15 18:11:18,939][52833] Updated weights for policy 0, policy_version 85270 (0.0008) -[2023-10-15 18:11:19,311][52833] Updated weights for policy 0, policy_version 85280 (0.0008) -[2023-10-15 18:11:19,858][52866] Updated weights for policy 1, policy_version 85510 (0.0008) -[2023-10-15 18:11:20,223][52866] Updated weights for policy 1, policy_version 85520 (0.0007) -[2023-10-15 18:11:20,590][52866] Updated weights for policy 1, policy_version 85530 (0.0009) -[2023-10-15 18:11:23,113][52833] Updated weights for policy 0, policy_version 85290 (0.0009) -[2023-10-15 18:11:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174915584. Throughput: 0: 1806.0, 1: 1808.9. Samples: 43741010. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:23,441][51532] Avg episode reward: [(0, '74.090'), (1, '65.370')] -[2023-10-15 18:11:23,492][52833] Updated weights for policy 0, policy_version 85300 (0.0007) -[2023-10-15 18:11:23,849][52833] Updated weights for policy 0, policy_version 85310 (0.0008) -[2023-10-15 18:11:24,231][52866] Updated weights for policy 1, policy_version 85540 (0.0009) -[2023-10-15 18:11:24,602][52866] Updated weights for policy 1, policy_version 85550 (0.0008) -[2023-10-15 18:11:24,962][52866] Updated weights for policy 1, policy_version 85560 (0.0009) -[2023-10-15 18:11:27,583][52833] Updated weights for policy 0, policy_version 85320 (0.0008) -[2023-10-15 18:11:27,945][52833] Updated weights for policy 0, policy_version 85330 (0.0008) -[2023-10-15 18:11:28,317][52833] Updated weights for policy 0, policy_version 85340 (0.0007) -[2023-10-15 18:11:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174981120. Throughput: 0: 1813.9, 1: 1809.3. Samples: 43762750. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:28,442][51532] Avg episode reward: [(0, '75.790'), (1, '65.510')] -[2023-10-15 18:11:28,801][52866] Updated weights for policy 1, policy_version 85570 (0.0008) -[2023-10-15 18:11:29,204][52866] Updated weights for policy 1, policy_version 85580 (0.0008) -[2023-10-15 18:11:29,570][52866] Updated weights for policy 1, policy_version 85590 (0.0010) -[2023-10-15 18:11:29,927][52866] Updated weights for policy 1, policy_version 85600 (0.0009) -[2023-10-15 18:11:32,022][52833] Updated weights for policy 0, policy_version 85350 (0.0007) -[2023-10-15 18:11:32,395][52833] Updated weights for policy 0, policy_version 85360 (0.0008) -[2023-10-15 18:11:32,771][52833] Updated weights for policy 0, policy_version 85370 (0.0008) -[2023-10-15 18:11:33,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 175079424. Throughput: 0: 1808.7, 1: 1805.1. Samples: 43773140. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:33,441][51532] Avg episode reward: [(0, '78.750'), (1, '65.760')] -[2023-10-15 18:11:33,549][52866] Updated weights for policy 1, policy_version 85610 (0.0008) -[2023-10-15 18:11:33,918][52866] Updated weights for policy 1, policy_version 85620 (0.0009) -[2023-10-15 18:11:34,288][52866] Updated weights for policy 1, policy_version 85630 (0.0007) -[2023-10-15 18:11:36,528][52833] Updated weights for policy 0, policy_version 85380 (0.0009) -[2023-10-15 18:11:36,896][52833] Updated weights for policy 0, policy_version 85390 (0.0009) -[2023-10-15 18:11:37,270][52833] Updated weights for policy 0, policy_version 85400 (0.0007) -[2023-10-15 18:11:38,183][52866] Updated weights for policy 1, policy_version 85640 (0.0009) -[2023-10-15 18:11:38,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 175144960. Throughput: 0: 1809.7, 1: 1800.1. Samples: 43794758. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:38,442][51532] Avg episode reward: [(0, '77.850'), (1, '63.310')] -[2023-10-15 18:11:38,545][52866] Updated weights for policy 1, policy_version 85650 (0.0007) -[2023-10-15 18:11:38,906][52866] Updated weights for policy 1, policy_version 85660 (0.0009) -[2023-10-15 18:11:40,941][52833] Updated weights for policy 0, policy_version 85410 (0.0009) -[2023-10-15 18:11:41,316][52833] Updated weights for policy 0, policy_version 85420 (0.0011) -[2023-10-15 18:11:41,688][52833] Updated weights for policy 0, policy_version 85430 (0.0009) -[2023-10-15 18:11:42,054][52833] Updated weights for policy 0, policy_version 85440 (0.0007) -[2023-10-15 18:11:42,794][52866] Updated weights for policy 1, policy_version 85670 (0.0010) -[2023-10-15 18:11:43,169][52866] Updated weights for policy 1, policy_version 85680 (0.0008) -[2023-10-15 18:11:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 175210496. Throughput: 0: 1796.1, 1: 1820.2. Samples: 43816208. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:43,442][51532] Avg episode reward: [(0, '79.580'), (1, '62.170')] -[2023-10-15 18:11:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000085440_87490560.pth... -[2023-10-15 18:11:43,489][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000083744_85753856.pth -[2023-10-15 18:11:43,542][52866] Updated weights for policy 1, policy_version 85690 (0.0009) -[2023-10-15 18:11:43,760][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000085696_87752704.pth... -[2023-10-15 18:11:43,788][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000084000_86016000.pth -[2023-10-15 18:11:45,885][52833] Updated weights for policy 0, policy_version 85450 (0.0008) -[2023-10-15 18:11:46,249][52833] Updated weights for policy 0, policy_version 85460 (0.0008) -[2023-10-15 18:11:46,615][52833] Updated weights for policy 0, policy_version 85470 (0.0010) -[2023-10-15 18:11:47,378][52866] Updated weights for policy 1, policy_version 85700 (0.0008) -[2023-10-15 18:11:47,740][52866] Updated weights for policy 1, policy_version 85710 (0.0008) -[2023-10-15 18:11:48,105][52866] Updated weights for policy 1, policy_version 85720 (0.0010) -[2023-10-15 18:11:48,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 175308800. Throughput: 0: 1808.3, 1: 1801.9. Samples: 43827444. Policy #0 lag: (min: 8.0, avg: 28.7, max: 40.0) -[2023-10-15 18:11:48,441][51532] Avg episode reward: [(0, '79.470'), (1, '62.400')] -[2023-10-15 18:11:50,229][52833] Updated weights for policy 0, policy_version 85480 (0.0007) -[2023-10-15 18:11:50,611][52833] Updated weights for policy 0, policy_version 85490 (0.0007) -[2023-10-15 18:11:50,977][52833] Updated weights for policy 0, policy_version 85500 (0.0007) -[2023-10-15 18:11:51,817][52866] Updated weights for policy 1, policy_version 85730 (0.0008) -[2023-10-15 18:11:52,179][52866] Updated weights for policy 1, policy_version 85740 (0.0007) -[2023-10-15 18:11:52,542][52866] Updated weights for policy 1, policy_version 85750 (0.0007) -[2023-10-15 18:11:52,900][52866] Updated weights for policy 1, policy_version 85760 (0.0007) -[2023-10-15 18:11:53,441][51532] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 175374336. Throughput: 0: 1794.1, 1: 1817.6. Samples: 43848774. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:11:53,441][51532] Avg episode reward: [(0, '81.540'), (1, '63.320')] -[2023-10-15 18:11:54,781][52833] Updated weights for policy 0, policy_version 85510 (0.0009) -[2023-10-15 18:11:55,153][52833] Updated weights for policy 0, policy_version 85520 (0.0009) -[2023-10-15 18:11:55,529][52833] Updated weights for policy 0, policy_version 85530 (0.0009) -[2023-10-15 18:11:56,594][52866] Updated weights for policy 1, policy_version 85770 (0.0009) -[2023-10-15 18:11:56,968][52866] Updated weights for policy 1, policy_version 85780 (0.0010) -[2023-10-15 18:11:57,343][52866] Updated weights for policy 1, policy_version 85790 (0.0009) -[2023-10-15 18:11:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 175439872. Throughput: 0: 1792.2, 1: 1795.0. Samples: 43870102. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:11:58,442][51532] Avg episode reward: [(0, '81.560'), (1, '60.940')] -[2023-10-15 18:11:59,222][52833] Updated weights for policy 0, policy_version 85540 (0.0008) -[2023-10-15 18:11:59,588][52833] Updated weights for policy 0, policy_version 85550 (0.0008) -[2023-10-15 18:11:59,965][52833] Updated weights for policy 0, policy_version 85560 (0.0009) -[2023-10-15 18:12:01,013][52866] Updated weights for policy 1, policy_version 85800 (0.0009) -[2023-10-15 18:12:01,375][52866] Updated weights for policy 1, policy_version 85810 (0.0009) -[2023-10-15 18:12:01,743][52866] Updated weights for policy 1, policy_version 85820 (0.0009) -[2023-10-15 18:12:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 175505408. Throughput: 0: 1792.7, 1: 1810.6. Samples: 43881160. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:03,442][51532] Avg episode reward: [(0, '78.760'), (1, '62.270')] -[2023-10-15 18:12:03,743][52833] Updated weights for policy 0, policy_version 85570 (0.0010) -[2023-10-15 18:12:04,113][52833] Updated weights for policy 0, policy_version 85580 (0.0009) -[2023-10-15 18:12:04,486][52833] Updated weights for policy 0, policy_version 85590 (0.0007) -[2023-10-15 18:12:04,859][52833] Updated weights for policy 0, policy_version 85600 (0.0007) -[2023-10-15 18:12:05,379][52866] Updated weights for policy 1, policy_version 85830 (0.0008) -[2023-10-15 18:12:05,745][52866] Updated weights for policy 1, policy_version 85840 (0.0009) -[2023-10-15 18:12:06,112][52866] Updated weights for policy 1, policy_version 85850 (0.0009) -[2023-10-15 18:12:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 175570944. Throughput: 0: 1792.0, 1: 1797.1. Samples: 43902522. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:08,442][51532] Avg episode reward: [(0, '81.060'), (1, '60.700')] -[2023-10-15 18:12:08,799][52833] Updated weights for policy 0, policy_version 85610 (0.0009) -[2023-10-15 18:12:09,185][52833] Updated weights for policy 0, policy_version 85620 (0.0008) -[2023-10-15 18:12:09,553][52833] Updated weights for policy 0, policy_version 85630 (0.0009) -[2023-10-15 18:12:09,738][52866] Updated weights for policy 1, policy_version 85860 (0.0011) -[2023-10-15 18:12:10,102][52866] Updated weights for policy 1, policy_version 85870 (0.0008) -[2023-10-15 18:12:10,463][52866] Updated weights for policy 1, policy_version 85880 (0.0010) -[2023-10-15 18:12:13,291][52833] Updated weights for policy 0, policy_version 85640 (0.0008) -[2023-10-15 18:12:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 175636480. Throughput: 0: 1806.7, 1: 1799.9. Samples: 43925048. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:13,441][51532] Avg episode reward: [(0, '79.260'), (1, '59.980')] -[2023-10-15 18:12:13,661][52833] Updated weights for policy 0, policy_version 85650 (0.0011) -[2023-10-15 18:12:14,025][52833] Updated weights for policy 0, policy_version 85660 (0.0011) -[2023-10-15 18:12:14,298][52866] Updated weights for policy 1, policy_version 85890 (0.0007) -[2023-10-15 18:12:14,693][52866] Updated weights for policy 1, policy_version 85900 (0.0007) -[2023-10-15 18:12:15,049][52866] Updated weights for policy 1, policy_version 85910 (0.0011) -[2023-10-15 18:12:15,418][52866] Updated weights for policy 1, policy_version 85920 (0.0008) -[2023-10-15 18:12:17,944][52833] Updated weights for policy 0, policy_version 85670 (0.0009) -[2023-10-15 18:12:18,312][52833] Updated weights for policy 0, policy_version 85680 (0.0009) -[2023-10-15 18:12:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 175702016. Throughput: 0: 1787.6, 1: 1802.4. Samples: 43934692. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:18,441][51532] Avg episode reward: [(0, '81.360'), (1, '60.610')] -[2023-10-15 18:12:18,682][52833] Updated weights for policy 0, policy_version 85690 (0.0008) -[2023-10-15 18:12:18,977][52866] Updated weights for policy 1, policy_version 85930 (0.0007) -[2023-10-15 18:12:19,341][52866] Updated weights for policy 1, policy_version 85940 (0.0009) -[2023-10-15 18:12:19,710][52866] Updated weights for policy 1, policy_version 85950 (0.0008) -[2023-10-15 18:12:22,431][52833] Updated weights for policy 0, policy_version 85700 (0.0008) -[2023-10-15 18:12:22,804][52833] Updated weights for policy 0, policy_version 85710 (0.0011) -[2023-10-15 18:12:23,166][52833] Updated weights for policy 0, policy_version 85720 (0.0008) -[2023-10-15 18:12:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 175767552. Throughput: 0: 1793.0, 1: 1804.2. Samples: 43956634. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:23,441][51532] Avg episode reward: [(0, '80.710'), (1, '62.670')] -[2023-10-15 18:12:23,484][52866] Updated weights for policy 1, policy_version 85960 (0.0008) -[2023-10-15 18:12:23,852][52866] Updated weights for policy 1, policy_version 85970 (0.0008) -[2023-10-15 18:12:24,218][52866] Updated weights for policy 1, policy_version 85980 (0.0007) -[2023-10-15 18:12:26,848][52833] Updated weights for policy 0, policy_version 85730 (0.0009) -[2023-10-15 18:12:27,222][52833] Updated weights for policy 0, policy_version 85740 (0.0009) -[2023-10-15 18:12:27,588][52833] Updated weights for policy 0, policy_version 85750 (0.0010) -[2023-10-15 18:12:27,801][52866] Updated weights for policy 1, policy_version 85990 (0.0008) -[2023-10-15 18:12:27,957][52833] Updated weights for policy 0, policy_version 85760 (0.0009) -[2023-10-15 18:12:28,171][52866] Updated weights for policy 1, policy_version 86000 (0.0007) -[2023-10-15 18:12:28,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 175865856. Throughput: 0: 1784.6, 1: 1806.1. Samples: 43977790. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:28,442][51532] Avg episode reward: [(0, '81.830'), (1, '63.090')] -[2023-10-15 18:12:28,541][52866] Updated weights for policy 1, policy_version 86010 (0.0008) -[2023-10-15 18:12:31,715][52833] Updated weights for policy 0, policy_version 85770 (0.0008) -[2023-10-15 18:12:32,083][52833] Updated weights for policy 0, policy_version 85780 (0.0008) -[2023-10-15 18:12:32,250][52866] Updated weights for policy 1, policy_version 86020 (0.0009) -[2023-10-15 18:12:32,453][52833] Updated weights for policy 0, policy_version 85790 (0.0007) -[2023-10-15 18:12:32,615][52866] Updated weights for policy 1, policy_version 86030 (0.0009) -[2023-10-15 18:12:32,984][52866] Updated weights for policy 1, policy_version 86040 (0.0007) -[2023-10-15 18:12:33,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 175964160. Throughput: 0: 1788.8, 1: 1808.9. Samples: 43989340. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:33,441][51532] Avg episode reward: [(0, '80.550'), (1, '61.610')] -[2023-10-15 18:12:36,226][52833] Updated weights for policy 0, policy_version 85800 (0.0007) -[2023-10-15 18:12:36,594][52833] Updated weights for policy 0, policy_version 85810 (0.0009) -[2023-10-15 18:12:36,853][52866] Updated weights for policy 1, policy_version 86050 (0.0008) -[2023-10-15 18:12:36,967][52833] Updated weights for policy 0, policy_version 85820 (0.0007) -[2023-10-15 18:12:37,219][52866] Updated weights for policy 1, policy_version 86060 (0.0008) -[2023-10-15 18:12:37,586][52866] Updated weights for policy 1, policy_version 86070 (0.0008) -[2023-10-15 18:12:37,944][52866] Updated weights for policy 1, policy_version 86080 (0.0009) -[2023-10-15 18:12:38,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 176029696. Throughput: 0: 1781.6, 1: 1808.8. Samples: 44010344. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:38,442][51532] Avg episode reward: [(0, '75.990'), (1, '66.100')] -[2023-10-15 18:12:40,587][52833] Updated weights for policy 0, policy_version 85830 (0.0008) -[2023-10-15 18:12:40,950][52833] Updated weights for policy 0, policy_version 85840 (0.0007) -[2023-10-15 18:12:41,323][52833] Updated weights for policy 0, policy_version 85850 (0.0007) -[2023-10-15 18:12:41,682][52866] Updated weights for policy 1, policy_version 86090 (0.0008) -[2023-10-15 18:12:42,040][52866] Updated weights for policy 1, policy_version 86100 (0.0008) -[2023-10-15 18:12:42,412][52866] Updated weights for policy 1, policy_version 86110 (0.0008) -[2023-10-15 18:12:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 176095232. Throughput: 0: 1779.1, 1: 1811.6. Samples: 44031686. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-15 18:12:43,442][51532] Avg episode reward: [(0, '78.180'), (1, '63.340')] -[2023-10-15 18:12:45,131][52833] Updated weights for policy 0, policy_version 85860 (0.0007) -[2023-10-15 18:12:45,499][52833] Updated weights for policy 0, policy_version 85870 (0.0008) -[2023-10-15 18:12:45,863][52833] Updated weights for policy 0, policy_version 85880 (0.0007) -[2023-10-15 18:12:46,165][52866] Updated weights for policy 1, policy_version 86120 (0.0008) -[2023-10-15 18:12:46,529][52866] Updated weights for policy 1, policy_version 86130 (0.0008) -[2023-10-15 18:12:46,894][52866] Updated weights for policy 1, policy_version 86140 (0.0007) -[2023-10-15 18:12:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176160768. Throughput: 0: 1785.6, 1: 1815.7. Samples: 44043218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:12:48,441][51532] Avg episode reward: [(0, '78.090'), (1, '63.930')] -[2023-10-15 18:12:49,518][52833] Updated weights for policy 0, policy_version 85890 (0.0009) -[2023-10-15 18:12:49,889][52833] Updated weights for policy 0, policy_version 85900 (0.0008) -[2023-10-15 18:12:50,254][52833] Updated weights for policy 0, policy_version 85910 (0.0009) -[2023-10-15 18:12:50,615][52833] Updated weights for policy 0, policy_version 85920 (0.0009) -[2023-10-15 18:12:50,702][52866] Updated weights for policy 1, policy_version 86150 (0.0007) -[2023-10-15 18:12:51,073][52866] Updated weights for policy 1, policy_version 86160 (0.0008) -[2023-10-15 18:12:51,447][52866] Updated weights for policy 1, policy_version 86170 (0.0009) -[2023-10-15 18:12:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 176226304. Throughput: 0: 1779.3, 1: 1805.8. Samples: 44063850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:12:53,442][51532] Avg episode reward: [(0, '79.750'), (1, '63.270')] -[2023-10-15 18:12:54,401][52833] Updated weights for policy 0, policy_version 85930 (0.0007) -[2023-10-15 18:12:54,765][52833] Updated weights for policy 0, policy_version 85940 (0.0009) -[2023-10-15 18:12:55,024][52866] Updated weights for policy 1, policy_version 86180 (0.0007) -[2023-10-15 18:12:55,134][52833] Updated weights for policy 0, policy_version 85950 (0.0008) -[2023-10-15 18:12:55,399][52866] Updated weights for policy 1, policy_version 86190 (0.0009) -[2023-10-15 18:12:55,762][52866] Updated weights for policy 1, policy_version 86200 (0.0009) -[2023-10-15 18:12:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176291840. Throughput: 0: 1788.9, 1: 1804.4. Samples: 44086746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:12:58,441][51532] Avg episode reward: [(0, '82.010'), (1, '67.150')] -[2023-10-15 18:12:58,760][52833] Updated weights for policy 0, policy_version 85960 (0.0010) -[2023-10-15 18:12:59,142][52833] Updated weights for policy 0, policy_version 85970 (0.0010) -[2023-10-15 18:12:59,504][52833] Updated weights for policy 0, policy_version 85980 (0.0007) -[2023-10-15 18:12:59,609][52866] Updated weights for policy 1, policy_version 86210 (0.0009) -[2023-10-15 18:13:00,005][52866] Updated weights for policy 1, policy_version 86220 (0.0009) -[2023-10-15 18:13:00,362][52866] Updated weights for policy 1, policy_version 86230 (0.0009) -[2023-10-15 18:13:00,731][52866] Updated weights for policy 1, policy_version 86240 (0.0008) -[2023-10-15 18:13:03,272][52833] Updated weights for policy 0, policy_version 85990 (0.0007) -[2023-10-15 18:13:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176357376. Throughput: 0: 1795.9, 1: 1801.6. Samples: 44096578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:13:03,442][51532] Avg episode reward: [(0, '77.770'), (1, '66.880')] -[2023-10-15 18:13:03,635][52833] Updated weights for policy 0, policy_version 86000 (0.0011) -[2023-10-15 18:13:04,008][52833] Updated weights for policy 0, policy_version 86010 (0.0007) -[2023-10-15 18:13:04,578][52866] Updated weights for policy 1, policy_version 86250 (0.0008) -[2023-10-15 18:13:04,943][52866] Updated weights for policy 1, policy_version 86260 (0.0007) -[2023-10-15 18:13:05,298][52866] Updated weights for policy 1, policy_version 86270 (0.0007) -[2023-10-15 18:13:07,738][52833] Updated weights for policy 0, policy_version 86020 (0.0007) -[2023-10-15 18:13:08,099][52833] Updated weights for policy 0, policy_version 86030 (0.0007) -[2023-10-15 18:13:08,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 176422912. Throughput: 0: 1807.0, 1: 1801.8. Samples: 44119030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:13:08,442][51532] Avg episode reward: [(0, '78.330'), (1, '65.650')] -[2023-10-15 18:13:08,479][52833] Updated weights for policy 0, policy_version 86040 (0.0007) -[2023-10-15 18:13:09,027][52866] Updated weights for policy 1, policy_version 86280 (0.0007) -[2023-10-15 18:13:09,385][52866] Updated weights for policy 1, policy_version 86290 (0.0007) -[2023-10-15 18:13:09,749][52866] Updated weights for policy 1, policy_version 86300 (0.0008) -[2023-10-15 18:13:12,275][52833] Updated weights for policy 0, policy_version 86050 (0.0007) -[2023-10-15 18:13:12,653][52833] Updated weights for policy 0, policy_version 86060 (0.0008) -[2023-10-15 18:13:13,020][52833] Updated weights for policy 0, policy_version 86070 (0.0008) -[2023-10-15 18:13:13,390][52833] Updated weights for policy 0, policy_version 86080 (0.0007) -[2023-10-15 18:13:13,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 176521216. Throughput: 0: 1811.8, 1: 1807.6. Samples: 44140666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:13:13,442][51532] Avg episode reward: [(0, '78.960'), (1, '69.380')] -[2023-10-15 18:13:13,520][52866] Updated weights for policy 1, policy_version 86310 (0.0008) -[2023-10-15 18:13:13,877][52866] Updated weights for policy 1, policy_version 86320 (0.0009) -[2023-10-15 18:13:14,243][52866] Updated weights for policy 1, policy_version 86330 (0.0010) -[2023-10-15 18:13:17,152][52833] Updated weights for policy 0, policy_version 86090 (0.0009) -[2023-10-15 18:13:17,519][52833] Updated weights for policy 0, policy_version 86100 (0.0011) -[2023-10-15 18:13:17,894][52833] Updated weights for policy 0, policy_version 86110 (0.0008) -[2023-10-15 18:13:17,980][52866] Updated weights for policy 1, policy_version 86340 (0.0010) -[2023-10-15 18:13:18,339][52866] Updated weights for policy 1, policy_version 86350 (0.0007) -[2023-10-15 18:13:18,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 176586752. Throughput: 0: 1804.0, 1: 1797.9. Samples: 44151424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:13:18,442][51532] Avg episode reward: [(0, '78.740'), (1, '65.670')] -[2023-10-15 18:13:18,710][52866] Updated weights for policy 1, policy_version 86360 (0.0008) -[2023-10-15 18:13:21,638][52833] Updated weights for policy 0, policy_version 86120 (0.0010) -[2023-10-15 18:13:22,013][52833] Updated weights for policy 0, policy_version 86130 (0.0011) -[2023-10-15 18:13:22,388][52833] Updated weights for policy 0, policy_version 86140 (0.0008) -[2023-10-15 18:13:22,446][52866] Updated weights for policy 1, policy_version 86370 (0.0007) -[2023-10-15 18:13:22,810][52866] Updated weights for policy 1, policy_version 86380 (0.0010) -[2023-10-15 18:13:23,171][52866] Updated weights for policy 1, policy_version 86390 (0.0010) -[2023-10-15 18:13:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 176652288. Throughput: 0: 1815.6, 1: 1803.6. Samples: 44173206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:13:23,442][51532] Avg episode reward: [(0, '77.860'), (1, '66.590')] -[2023-10-15 18:13:23,548][52866] Updated weights for policy 1, policy_version 86400 (0.0010) -[2023-10-15 18:13:26,047][52833] Updated weights for policy 0, policy_version 86150 (0.0007) -[2023-10-15 18:13:26,418][52833] Updated weights for policy 0, policy_version 86160 (0.0008) -[2023-10-15 18:13:26,791][52833] Updated weights for policy 0, policy_version 86170 (0.0008) -[2023-10-15 18:13:27,240][52866] Updated weights for policy 1, policy_version 86410 (0.0009) -[2023-10-15 18:13:27,608][52866] Updated weights for policy 1, policy_version 86420 (0.0007) -[2023-10-15 18:13:27,974][52866] Updated weights for policy 1, policy_version 86430 (0.0009) -[2023-10-15 18:13:28,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 176750592. Throughput: 0: 1801.6, 1: 1798.9. Samples: 44193712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:13:28,442][51532] Avg episode reward: [(0, '77.560'), (1, '67.100')] -[2023-10-15 18:13:30,368][52833] Updated weights for policy 0, policy_version 86180 (0.0009) -[2023-10-15 18:13:30,733][52833] Updated weights for policy 0, policy_version 86190 (0.0007) -[2023-10-15 18:13:31,105][52833] Updated weights for policy 0, policy_version 86200 (0.0011) -[2023-10-15 18:13:31,732][52866] Updated weights for policy 1, policy_version 86440 (0.0008) -[2023-10-15 18:13:32,106][52866] Updated weights for policy 1, policy_version 86450 (0.0007) -[2023-10-15 18:13:32,463][52866] Updated weights for policy 1, policy_version 86460 (0.0008) -[2023-10-15 18:13:33,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176816128. Throughput: 0: 1814.9, 1: 1796.7. Samples: 44205742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:13:33,441][51532] Avg episode reward: [(0, '78.190'), (1, '67.810')] -[2023-10-15 18:13:34,770][52833] Updated weights for policy 0, policy_version 86210 (0.0009) -[2023-10-15 18:13:35,138][52833] Updated weights for policy 0, policy_version 86220 (0.0009) -[2023-10-15 18:13:35,500][52833] Updated weights for policy 0, policy_version 86230 (0.0009) -[2023-10-15 18:13:35,863][52833] Updated weights for policy 0, policy_version 86240 (0.0010) -[2023-10-15 18:13:36,149][52866] Updated weights for policy 1, policy_version 86470 (0.0009) -[2023-10-15 18:13:36,520][52866] Updated weights for policy 1, policy_version 86480 (0.0007) -[2023-10-15 18:13:36,893][52866] Updated weights for policy 1, policy_version 86490 (0.0007) -[2023-10-15 18:13:38,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176881664. Throughput: 0: 1812.5, 1: 1803.9. Samples: 44226590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:13:38,441][51532] Avg episode reward: [(0, '80.460'), (1, '63.310')] -[2023-10-15 18:13:39,749][52833] Updated weights for policy 0, policy_version 86250 (0.0009) -[2023-10-15 18:13:40,124][52833] Updated weights for policy 0, policy_version 86260 (0.0007) -[2023-10-15 18:13:40,487][52833] Updated weights for policy 0, policy_version 86270 (0.0008) -[2023-10-15 18:13:40,488][52866] Updated weights for policy 1, policy_version 86500 (0.0007) -[2023-10-15 18:13:40,861][52866] Updated weights for policy 1, policy_version 86510 (0.0007) -[2023-10-15 18:13:41,225][52866] Updated weights for policy 1, policy_version 86520 (0.0008) -[2023-10-15 18:13:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 176947200. Throughput: 0: 1805.9, 1: 1800.1. Samples: 44249014. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:13:43,442][51532] Avg episode reward: [(0, '83.530'), (1, '65.630')] -[2023-10-15 18:13:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000086528_88604672.pth... -[2023-10-15 18:13:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000086272_88342528.pth... -[2023-10-15 18:13:43,491][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000084608_86638592.pth -[2023-10-15 18:13:43,497][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000084832_86867968.pth -[2023-10-15 18:13:44,294][52833] Updated weights for policy 0, policy_version 86280 (0.0009) -[2023-10-15 18:13:44,666][52833] Updated weights for policy 0, policy_version 86290 (0.0010) -[2023-10-15 18:13:44,959][52866] Updated weights for policy 1, policy_version 86530 (0.0011) -[2023-10-15 18:13:45,034][52833] Updated weights for policy 0, policy_version 86300 (0.0007) -[2023-10-15 18:13:45,321][52866] Updated weights for policy 1, policy_version 86540 (0.0008) -[2023-10-15 18:13:45,689][52866] Updated weights for policy 1, policy_version 86550 (0.0009) -[2023-10-15 18:13:46,060][52866] Updated weights for policy 1, policy_version 86560 (0.0007) -[2023-10-15 18:13:48,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177012736. Throughput: 0: 1796.7, 1: 1810.8. Samples: 44258916. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:13:48,441][51532] Avg episode reward: [(0, '85.910'), (1, '65.550')] -[2023-10-15 18:13:48,442][52410] Saving new best policy, reward=85.910! -[2023-10-15 18:13:48,855][52833] Updated weights for policy 0, policy_version 86310 (0.0009) -[2023-10-15 18:13:49,227][52833] Updated weights for policy 0, policy_version 86320 (0.0011) -[2023-10-15 18:13:49,599][52833] Updated weights for policy 0, policy_version 86330 (0.0008) -[2023-10-15 18:13:49,796][52866] Updated weights for policy 1, policy_version 86570 (0.0008) -[2023-10-15 18:13:50,163][52866] Updated weights for policy 1, policy_version 86580 (0.0008) -[2023-10-15 18:13:50,526][52866] Updated weights for policy 1, policy_version 86590 (0.0008) -[2023-10-15 18:13:53,216][52833] Updated weights for policy 0, policy_version 86340 (0.0008) -[2023-10-15 18:13:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177078272. Throughput: 0: 1798.6, 1: 1808.5. Samples: 44281350. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:13:53,441][51532] Avg episode reward: [(0, '82.460'), (1, '70.920')] -[2023-10-15 18:13:53,579][52833] Updated weights for policy 0, policy_version 86350 (0.0011) -[2023-10-15 18:13:53,944][52833] Updated weights for policy 0, policy_version 86360 (0.0009) -[2023-10-15 18:13:54,098][52866] Updated weights for policy 1, policy_version 86600 (0.0008) -[2023-10-15 18:13:54,465][52866] Updated weights for policy 1, policy_version 86610 (0.0008) -[2023-10-15 18:13:54,836][52866] Updated weights for policy 1, policy_version 86620 (0.0010) -[2023-10-15 18:13:57,617][52833] Updated weights for policy 0, policy_version 86370 (0.0010) -[2023-10-15 18:13:57,986][52833] Updated weights for policy 0, policy_version 86380 (0.0008) -[2023-10-15 18:13:58,355][52833] Updated weights for policy 0, policy_version 86390 (0.0008) -[2023-10-15 18:13:58,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 177143808. Throughput: 0: 1809.8, 1: 1814.4. Samples: 44303756. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:13:58,442][51532] Avg episode reward: [(0, '84.580'), (1, '72.020')] -[2023-10-15 18:13:58,545][52866] Updated weights for policy 1, policy_version 86630 (0.0008) -[2023-10-15 18:13:58,725][52833] Updated weights for policy 0, policy_version 86400 (0.0009) -[2023-10-15 18:13:58,911][52866] Updated weights for policy 1, policy_version 86640 (0.0008) -[2023-10-15 18:13:59,279][52866] Updated weights for policy 1, policy_version 86650 (0.0010) -[2023-10-15 18:14:02,620][52833] Updated weights for policy 0, policy_version 86410 (0.0008) -[2023-10-15 18:14:02,941][52866] Updated weights for policy 1, policy_version 86660 (0.0008) -[2023-10-15 18:14:02,974][52833] Updated weights for policy 0, policy_version 86420 (0.0008) -[2023-10-15 18:14:03,313][52866] Updated weights for policy 1, policy_version 86670 (0.0008) -[2023-10-15 18:14:03,334][52833] Updated weights for policy 0, policy_version 86430 (0.0007) -[2023-10-15 18:14:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 177242112. Throughput: 0: 1799.6, 1: 1813.6. Samples: 44314022. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:14:03,442][51532] Avg episode reward: [(0, '80.670'), (1, '69.100')] -[2023-10-15 18:14:03,682][52866] Updated weights for policy 1, policy_version 86680 (0.0008) -[2023-10-15 18:14:07,079][52833] Updated weights for policy 0, policy_version 86440 (0.0010) -[2023-10-15 18:14:07,410][52866] Updated weights for policy 1, policy_version 86690 (0.0008) -[2023-10-15 18:14:07,454][52833] Updated weights for policy 0, policy_version 86450 (0.0008) -[2023-10-15 18:14:07,770][52866] Updated weights for policy 1, policy_version 86700 (0.0010) -[2023-10-15 18:14:07,822][52833] Updated weights for policy 0, policy_version 86460 (0.0007) -[2023-10-15 18:14:08,132][52866] Updated weights for policy 1, policy_version 86710 (0.0007) -[2023-10-15 18:14:08,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 177307648. Throughput: 0: 1811.4, 1: 1813.0. Samples: 44336304. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:14:08,442][51532] Avg episode reward: [(0, '81.550'), (1, '70.130')] -[2023-10-15 18:14:08,491][52866] Updated weights for policy 1, policy_version 86720 (0.0007) -[2023-10-15 18:14:11,479][52833] Updated weights for policy 0, policy_version 86470 (0.0007) -[2023-10-15 18:14:11,842][52833] Updated weights for policy 0, policy_version 86480 (0.0007) -[2023-10-15 18:14:12,213][52833] Updated weights for policy 0, policy_version 86490 (0.0008) -[2023-10-15 18:14:12,308][52866] Updated weights for policy 1, policy_version 86730 (0.0007) -[2023-10-15 18:14:12,679][52866] Updated weights for policy 1, policy_version 86740 (0.0007) -[2023-10-15 18:14:13,048][52866] Updated weights for policy 1, policy_version 86750 (0.0008) -[2023-10-15 18:14:13,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 177405952. Throughput: 0: 1794.7, 1: 1814.1. Samples: 44356106. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:14:13,441][51532] Avg episode reward: [(0, '83.670'), (1, '66.980')] -[2023-10-15 18:14:16,016][52833] Updated weights for policy 0, policy_version 86500 (0.0008) -[2023-10-15 18:14:16,375][52833] Updated weights for policy 0, policy_version 86510 (0.0007) -[2023-10-15 18:14:16,738][52833] Updated weights for policy 0, policy_version 86520 (0.0007) -[2023-10-15 18:14:16,796][52866] Updated weights for policy 1, policy_version 86760 (0.0008) -[2023-10-15 18:14:17,168][52866] Updated weights for policy 1, policy_version 86770 (0.0007) -[2023-10-15 18:14:17,535][52866] Updated weights for policy 1, policy_version 86780 (0.0010) -[2023-10-15 18:14:18,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 177471488. Throughput: 0: 1806.4, 1: 1810.5. Samples: 44368504. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:14:18,442][51532] Avg episode reward: [(0, '84.290'), (1, '68.810')] -[2023-10-15 18:14:20,538][52833] Updated weights for policy 0, policy_version 86530 (0.0007) -[2023-10-15 18:14:20,907][52833] Updated weights for policy 0, policy_version 86540 (0.0007) -[2023-10-15 18:14:21,284][52833] Updated weights for policy 0, policy_version 86550 (0.0008) -[2023-10-15 18:14:21,387][52866] Updated weights for policy 1, policy_version 86790 (0.0010) -[2023-10-15 18:14:21,646][52833] Updated weights for policy 0, policy_version 86560 (0.0007) -[2023-10-15 18:14:21,754][52866] Updated weights for policy 1, policy_version 86800 (0.0009) -[2023-10-15 18:14:22,127][52866] Updated weights for policy 1, policy_version 86810 (0.0009) -[2023-10-15 18:14:23,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 177537024. Throughput: 0: 1783.5, 1: 1812.1. Samples: 44388394. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:14:23,442][51532] Avg episode reward: [(0, '84.530'), (1, '69.160')] -[2023-10-15 18:14:25,399][52833] Updated weights for policy 0, policy_version 86570 (0.0007) -[2023-10-15 18:14:25,777][52833] Updated weights for policy 0, policy_version 86580 (0.0011) -[2023-10-15 18:14:25,907][52866] Updated weights for policy 1, policy_version 86820 (0.0007) -[2023-10-15 18:14:26,149][52833] Updated weights for policy 0, policy_version 86590 (0.0008) -[2023-10-15 18:14:26,275][52866] Updated weights for policy 1, policy_version 86830 (0.0007) -[2023-10-15 18:14:26,641][52866] Updated weights for policy 1, policy_version 86840 (0.0009) -[2023-10-15 18:14:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177602560. Throughput: 0: 1783.9, 1: 1805.3. Samples: 44410526. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:14:28,442][51532] Avg episode reward: [(0, '86.710'), (1, '66.990')] -[2023-10-15 18:14:28,454][52410] Saving new best policy, reward=86.710! -[2023-10-15 18:14:30,012][52833] Updated weights for policy 0, policy_version 86600 (0.0007) -[2023-10-15 18:14:30,376][52833] Updated weights for policy 0, policy_version 86610 (0.0007) -[2023-10-15 18:14:30,491][52866] Updated weights for policy 1, policy_version 86850 (0.0010) -[2023-10-15 18:14:30,747][52833] Updated weights for policy 0, policy_version 86620 (0.0008) -[2023-10-15 18:14:30,887][52866] Updated weights for policy 1, policy_version 86860 (0.0007) -[2023-10-15 18:14:31,249][52866] Updated weights for policy 1, policy_version 86870 (0.0010) -[2023-10-15 18:14:31,616][52866] Updated weights for policy 1, policy_version 86880 (0.0008) -[2023-10-15 18:14:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177668096. Throughput: 0: 1788.1, 1: 1817.7. Samples: 44421178. Policy #0 lag: (min: 13.0, avg: 26.9, max: 45.0) -[2023-10-15 18:14:33,441][51532] Avg episode reward: [(0, '83.790'), (1, '67.990')] -[2023-10-15 18:14:34,410][52833] Updated weights for policy 0, policy_version 86630 (0.0009) -[2023-10-15 18:14:34,772][52833] Updated weights for policy 0, policy_version 86640 (0.0008) -[2023-10-15 18:14:35,147][52833] Updated weights for policy 0, policy_version 86650 (0.0007) -[2023-10-15 18:14:35,379][52866] Updated weights for policy 1, policy_version 86890 (0.0007) -[2023-10-15 18:14:35,745][52866] Updated weights for policy 1, policy_version 86900 (0.0008) -[2023-10-15 18:14:36,110][52866] Updated weights for policy 1, policy_version 86910 (0.0010) -[2023-10-15 18:14:38,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 177733632. Throughput: 0: 1783.3, 1: 1797.7. Samples: 44442496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:14:38,442][51532] Avg episode reward: [(0, '79.420'), (1, '67.250')] -[2023-10-15 18:14:38,820][52833] Updated weights for policy 0, policy_version 86660 (0.0009) -[2023-10-15 18:14:39,190][52833] Updated weights for policy 0, policy_version 86670 (0.0010) -[2023-10-15 18:14:39,556][52833] Updated weights for policy 0, policy_version 86680 (0.0009) -[2023-10-15 18:14:39,783][52866] Updated weights for policy 1, policy_version 86920 (0.0008) -[2023-10-15 18:14:40,145][52866] Updated weights for policy 1, policy_version 86930 (0.0008) -[2023-10-15 18:14:40,516][52866] Updated weights for policy 1, policy_version 86940 (0.0010) -[2023-10-15 18:14:43,351][52833] Updated weights for policy 0, policy_version 86690 (0.0009) -[2023-10-15 18:14:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177799168. Throughput: 0: 1792.7, 1: 1795.7. Samples: 44465234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:14:43,441][51532] Avg episode reward: [(0, '76.340'), (1, '72.340')] -[2023-10-15 18:14:43,729][52833] Updated weights for policy 0, policy_version 86700 (0.0007) -[2023-10-15 18:14:44,101][52833] Updated weights for policy 0, policy_version 86710 (0.0008) -[2023-10-15 18:14:44,108][52866] Updated weights for policy 1, policy_version 86950 (0.0007) -[2023-10-15 18:14:44,463][52833] Updated weights for policy 0, policy_version 86720 (0.0008) -[2023-10-15 18:14:44,479][52866] Updated weights for policy 1, policy_version 86960 (0.0008) -[2023-10-15 18:14:44,841][52866] Updated weights for policy 1, policy_version 86970 (0.0008) -[2023-10-15 18:14:48,331][52833] Updated weights for policy 0, policy_version 86730 (0.0007) -[2023-10-15 18:14:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 177864704. Throughput: 0: 1780.3, 1: 1799.8. Samples: 44475126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:14:48,441][51532] Avg episode reward: [(0, '76.470'), (1, '66.050')] -[2023-10-15 18:14:48,584][52866] Updated weights for policy 1, policy_version 86980 (0.0008) -[2023-10-15 18:14:48,704][52833] Updated weights for policy 0, policy_version 86740 (0.0008) -[2023-10-15 18:14:48,956][52866] Updated weights for policy 1, policy_version 86990 (0.0008) -[2023-10-15 18:14:49,077][52833] Updated weights for policy 0, policy_version 86750 (0.0007) -[2023-10-15 18:14:49,326][52866] Updated weights for policy 1, policy_version 87000 (0.0010) -[2023-10-15 18:14:52,949][52833] Updated weights for policy 0, policy_version 86760 (0.0008) -[2023-10-15 18:14:53,022][52866] Updated weights for policy 1, policy_version 87010 (0.0008) -[2023-10-15 18:14:53,316][52833] Updated weights for policy 0, policy_version 86770 (0.0009) -[2023-10-15 18:14:53,389][52866] Updated weights for policy 1, policy_version 87020 (0.0008) -[2023-10-15 18:14:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 177930240. Throughput: 0: 1782.0, 1: 1802.0. Samples: 44497586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:14:53,442][51532] Avg episode reward: [(0, '75.200'), (1, '65.930')] -[2023-10-15 18:14:53,676][52833] Updated weights for policy 0, policy_version 86780 (0.0010) -[2023-10-15 18:14:53,751][52866] Updated weights for policy 1, policy_version 87030 (0.0008) -[2023-10-15 18:14:54,112][52866] Updated weights for policy 1, policy_version 87040 (0.0011) -[2023-10-15 18:14:57,471][52833] Updated weights for policy 0, policy_version 86790 (0.0009) -[2023-10-15 18:14:57,749][52866] Updated weights for policy 1, policy_version 87050 (0.0009) -[2023-10-15 18:14:57,829][52833] Updated weights for policy 0, policy_version 86800 (0.0008) -[2023-10-15 18:14:58,113][52866] Updated weights for policy 1, policy_version 87060 (0.0008) -[2023-10-15 18:14:58,202][52833] Updated weights for policy 0, policy_version 86810 (0.0009) -[2023-10-15 18:14:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 178028544. Throughput: 0: 1793.7, 1: 1813.2. Samples: 44518420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:14:58,441][51532] Avg episode reward: [(0, '72.090'), (1, '66.570')] -[2023-10-15 18:14:58,486][52866] Updated weights for policy 1, policy_version 87070 (0.0007) -[2023-10-15 18:15:01,879][52833] Updated weights for policy 0, policy_version 86820 (0.0008) -[2023-10-15 18:15:02,244][52833] Updated weights for policy 0, policy_version 86830 (0.0008) -[2023-10-15 18:15:02,249][52866] Updated weights for policy 1, policy_version 87080 (0.0007) -[2023-10-15 18:15:02,616][52833] Updated weights for policy 0, policy_version 86840 (0.0007) -[2023-10-15 18:15:02,619][52866] Updated weights for policy 1, policy_version 87090 (0.0010) -[2023-10-15 18:15:02,991][52866] Updated weights for policy 1, policy_version 87100 (0.0011) -[2023-10-15 18:15:03,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 178126848. Throughput: 0: 1779.5, 1: 1803.1. Samples: 44529718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:15:03,442][51532] Avg episode reward: [(0, '73.860'), (1, '69.130')] -[2023-10-15 18:15:06,363][52833] Updated weights for policy 0, policy_version 86850 (0.0009) -[2023-10-15 18:15:06,696][52866] Updated weights for policy 1, policy_version 87110 (0.0010) -[2023-10-15 18:15:06,731][52833] Updated weights for policy 0, policy_version 86860 (0.0008) -[2023-10-15 18:15:07,058][52866] Updated weights for policy 1, policy_version 87120 (0.0009) -[2023-10-15 18:15:07,099][52833] Updated weights for policy 0, policy_version 86870 (0.0008) -[2023-10-15 18:15:07,417][52866] Updated weights for policy 1, policy_version 87130 (0.0008) -[2023-10-15 18:15:07,460][52833] Updated weights for policy 0, policy_version 86880 (0.0008) -[2023-10-15 18:15:08,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 178192384. Throughput: 0: 1794.9, 1: 1813.7. Samples: 44550780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:15:08,442][51532] Avg episode reward: [(0, '73.630'), (1, '70.030')] -[2023-10-15 18:15:11,179][52866] Updated weights for policy 1, policy_version 87140 (0.0009) -[2023-10-15 18:15:11,439][52833] Updated weights for policy 0, policy_version 86890 (0.0008) -[2023-10-15 18:15:11,553][52866] Updated weights for policy 1, policy_version 87150 (0.0007) -[2023-10-15 18:15:11,813][52833] Updated weights for policy 0, policy_version 86900 (0.0008) -[2023-10-15 18:15:11,906][52866] Updated weights for policy 1, policy_version 87160 (0.0007) -[2023-10-15 18:15:12,176][52833] Updated weights for policy 0, policy_version 86910 (0.0009) -[2023-10-15 18:15:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 178257920. Throughput: 0: 1774.9, 1: 1802.0. Samples: 44571486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:15:13,442][51532] Avg episode reward: [(0, '71.970'), (1, '69.850')] -[2023-10-15 18:15:15,825][52866] Updated weights for policy 1, policy_version 87170 (0.0007) -[2023-10-15 18:15:15,834][52833] Updated weights for policy 0, policy_version 86920 (0.0009) -[2023-10-15 18:15:16,198][52833] Updated weights for policy 0, policy_version 86930 (0.0009) -[2023-10-15 18:15:16,228][52866] Updated weights for policy 1, policy_version 87180 (0.0010) -[2023-10-15 18:15:16,569][52833] Updated weights for policy 0, policy_version 86940 (0.0008) -[2023-10-15 18:15:16,597][52866] Updated weights for policy 1, policy_version 87190 (0.0010) -[2023-10-15 18:15:16,957][52866] Updated weights for policy 1, policy_version 87200 (0.0009) -[2023-10-15 18:15:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 178323456. Throughput: 0: 1798.0, 1: 1804.8. Samples: 44583300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:15:18,442][51532] Avg episode reward: [(0, '73.040'), (1, '72.230')] -[2023-10-15 18:15:20,460][52833] Updated weights for policy 0, policy_version 86950 (0.0007) -[2023-10-15 18:15:20,686][52866] Updated weights for policy 1, policy_version 87210 (0.0008) -[2023-10-15 18:15:20,833][52833] Updated weights for policy 0, policy_version 86960 (0.0008) -[2023-10-15 18:15:21,054][52866] Updated weights for policy 1, policy_version 87220 (0.0008) -[2023-10-15 18:15:21,199][52833] Updated weights for policy 0, policy_version 86970 (0.0008) -[2023-10-15 18:15:21,409][52866] Updated weights for policy 1, policy_version 87230 (0.0009) -[2023-10-15 18:15:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 178388992. Throughput: 0: 1775.0, 1: 1797.3. Samples: 44603250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:15:23,442][51532] Avg episode reward: [(0, '70.200'), (1, '71.840')] -[2023-10-15 18:15:25,016][52833] Updated weights for policy 0, policy_version 86980 (0.0008) -[2023-10-15 18:15:25,267][52866] Updated weights for policy 1, policy_version 87240 (0.0008) -[2023-10-15 18:15:25,395][52833] Updated weights for policy 0, policy_version 86990 (0.0009) -[2023-10-15 18:15:25,639][52866] Updated weights for policy 1, policy_version 87250 (0.0008) -[2023-10-15 18:15:25,763][52833] Updated weights for policy 0, policy_version 87000 (0.0009) -[2023-10-15 18:15:26,011][52866] Updated weights for policy 1, policy_version 87260 (0.0007) -[2023-10-15 18:15:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 178454528. Throughput: 0: 1770.6, 1: 1786.6. Samples: 44625306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:15:28,441][51532] Avg episode reward: [(0, '72.150'), (1, '72.690')] -[2023-10-15 18:15:29,506][52833] Updated weights for policy 0, policy_version 87010 (0.0007) -[2023-10-15 18:15:29,833][52866] Updated weights for policy 1, policy_version 87270 (0.0008) -[2023-10-15 18:15:29,872][52833] Updated weights for policy 0, policy_version 87020 (0.0007) -[2023-10-15 18:15:30,197][52866] Updated weights for policy 1, policy_version 87280 (0.0010) -[2023-10-15 18:15:30,242][52833] Updated weights for policy 0, policy_version 87030 (0.0007) -[2023-10-15 18:15:30,565][52866] Updated weights for policy 1, policy_version 87290 (0.0007) -[2023-10-15 18:15:30,615][52833] Updated weights for policy 0, policy_version 87040 (0.0009) -[2023-10-15 18:15:33,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 178520064. Throughput: 0: 1775.1, 1: 1779.6. Samples: 44635084. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:15:33,442][51532] Avg episode reward: [(0, '71.360'), (1, '73.500')] -[2023-10-15 18:15:34,337][52833] Updated weights for policy 0, policy_version 87050 (0.0007) -[2023-10-15 18:15:34,339][52866] Updated weights for policy 1, policy_version 87300 (0.0009) -[2023-10-15 18:15:34,703][52866] Updated weights for policy 1, policy_version 87310 (0.0008) -[2023-10-15 18:15:34,706][52833] Updated weights for policy 0, policy_version 87060 (0.0008) -[2023-10-15 18:15:35,075][52866] Updated weights for policy 1, policy_version 87320 (0.0008) -[2023-10-15 18:15:35,078][52833] Updated weights for policy 0, policy_version 87070 (0.0008) -[2023-10-15 18:15:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 178585600. Throughput: 0: 1779.6, 1: 1776.0. Samples: 44657586. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:15:38,441][51532] Avg episode reward: [(0, '73.310'), (1, '74.260')] -[2023-10-15 18:15:38,707][52833] Updated weights for policy 0, policy_version 87080 (0.0007) -[2023-10-15 18:15:38,927][52866] Updated weights for policy 1, policy_version 87330 (0.0008) -[2023-10-15 18:15:39,073][52833] Updated weights for policy 0, policy_version 87090 (0.0007) -[2023-10-15 18:15:39,292][52866] Updated weights for policy 1, policy_version 87340 (0.0008) -[2023-10-15 18:15:39,450][52833] Updated weights for policy 0, policy_version 87100 (0.0009) -[2023-10-15 18:15:39,653][52866] Updated weights for policy 1, policy_version 87350 (0.0009) -[2023-10-15 18:15:40,025][52866] Updated weights for policy 1, policy_version 87360 (0.0008) -[2023-10-15 18:15:43,287][52833] Updated weights for policy 0, policy_version 87110 (0.0009) -[2023-10-15 18:15:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 178651136. Throughput: 0: 1797.9, 1: 1793.6. Samples: 44680038. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:15:43,441][51532] Avg episode reward: [(0, '72.800'), (1, '68.890')] -[2023-10-15 18:15:43,659][52833] Updated weights for policy 0, policy_version 87120 (0.0009) -[2023-10-15 18:15:43,724][52866] Updated weights for policy 1, policy_version 87370 (0.0009) -[2023-10-15 18:15:44,022][52833] Updated weights for policy 0, policy_version 87130 (0.0007) -[2023-10-15 18:15:44,086][52866] Updated weights for policy 1, policy_version 87380 (0.0009) -[2023-10-15 18:15:44,230][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000087136_89227264.pth... -[2023-10-15 18:15:44,259][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000085440_87490560.pth -[2023-10-15 18:15:44,439][52866] Updated weights for policy 1, policy_version 87390 (0.0008) -[2023-10-15 18:15:44,512][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000087392_89489408.pth... -[2023-10-15 18:15:44,541][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000085696_87752704.pth -[2023-10-15 18:15:47,770][52833] Updated weights for policy 0, policy_version 87140 (0.0007) -[2023-10-15 18:15:48,134][52833] Updated weights for policy 0, policy_version 87150 (0.0008) -[2023-10-15 18:15:48,183][52866] Updated weights for policy 1, policy_version 87400 (0.0008) -[2023-10-15 18:15:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 178716672. Throughput: 0: 1781.3, 1: 1776.3. Samples: 44689808. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:15:48,441][51532] Avg episode reward: [(0, '73.860'), (1, '70.570')] -[2023-10-15 18:15:48,496][52833] Updated weights for policy 0, policy_version 87160 (0.0009) -[2023-10-15 18:15:48,545][52866] Updated weights for policy 1, policy_version 87410 (0.0008) -[2023-10-15 18:15:48,907][52866] Updated weights for policy 1, policy_version 87420 (0.0007) -[2023-10-15 18:15:52,152][52833] Updated weights for policy 0, policy_version 87170 (0.0007) -[2023-10-15 18:15:52,520][52833] Updated weights for policy 0, policy_version 87180 (0.0007) -[2023-10-15 18:15:52,600][52866] Updated weights for policy 1, policy_version 87430 (0.0009) -[2023-10-15 18:15:52,893][52833] Updated weights for policy 0, policy_version 87190 (0.0009) -[2023-10-15 18:15:52,963][52866] Updated weights for policy 1, policy_version 87440 (0.0007) -[2023-10-15 18:15:53,261][52833] Updated weights for policy 0, policy_version 87200 (0.0007) -[2023-10-15 18:15:53,326][52866] Updated weights for policy 1, policy_version 87450 (0.0010) -[2023-10-15 18:15:53,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 178814976. Throughput: 0: 1802.1, 1: 1791.4. Samples: 44712488. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:15:53,442][51532] Avg episode reward: [(0, '69.810'), (1, '73.150')] -[2023-10-15 18:15:57,294][52866] Updated weights for policy 1, policy_version 87460 (0.0007) -[2023-10-15 18:15:57,310][52833] Updated weights for policy 0, policy_version 87210 (0.0007) -[2023-10-15 18:15:57,651][52866] Updated weights for policy 1, policy_version 87470 (0.0007) -[2023-10-15 18:15:57,671][52833] Updated weights for policy 0, policy_version 87220 (0.0007) -[2023-10-15 18:15:58,019][52866] Updated weights for policy 1, policy_version 87480 (0.0007) -[2023-10-15 18:15:58,045][52833] Updated weights for policy 0, policy_version 87230 (0.0008) -[2023-10-15 18:15:58,441][51532] Fps is (10 sec: 19660.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 178913280. Throughput: 0: 1793.6, 1: 1783.5. Samples: 44732452. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:15:58,442][51532] Avg episode reward: [(0, '72.640'), (1, '72.480')] -[2023-10-15 18:16:01,671][52833] Updated weights for policy 0, policy_version 87240 (0.0009) -[2023-10-15 18:16:01,953][52866] Updated weights for policy 1, policy_version 87490 (0.0008) -[2023-10-15 18:16:02,042][52833] Updated weights for policy 0, policy_version 87250 (0.0008) -[2023-10-15 18:16:02,337][52866] Updated weights for policy 1, policy_version 87500 (0.0008) -[2023-10-15 18:16:02,404][52833] Updated weights for policy 0, policy_version 87260 (0.0008) -[2023-10-15 18:16:02,701][52866] Updated weights for policy 1, policy_version 87510 (0.0010) -[2023-10-15 18:16:03,059][52866] Updated weights for policy 1, policy_version 87520 (0.0011) -[2023-10-15 18:16:03,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 178978816. Throughput: 0: 1795.3, 1: 1779.2. Samples: 44744152. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:16:03,441][51532] Avg episode reward: [(0, '72.530'), (1, '70.290')] -[2023-10-15 18:16:05,985][52833] Updated weights for policy 0, policy_version 87270 (0.0009) -[2023-10-15 18:16:06,354][52833] Updated weights for policy 0, policy_version 87280 (0.0008) -[2023-10-15 18:16:06,724][52833] Updated weights for policy 0, policy_version 87290 (0.0008) -[2023-10-15 18:16:06,847][52866] Updated weights for policy 1, policy_version 87530 (0.0007) -[2023-10-15 18:16:07,207][52866] Updated weights for policy 1, policy_version 87540 (0.0008) -[2023-10-15 18:16:07,579][52866] Updated weights for policy 1, policy_version 87550 (0.0010) -[2023-10-15 18:16:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 179044352. Throughput: 0: 1792.5, 1: 1791.5. Samples: 44764528. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:16:08,442][51532] Avg episode reward: [(0, '70.000'), (1, '69.020')] -[2023-10-15 18:16:10,538][52833] Updated weights for policy 0, policy_version 87300 (0.0007) -[2023-10-15 18:16:10,910][52833] Updated weights for policy 0, policy_version 87310 (0.0007) -[2023-10-15 18:16:11,278][52833] Updated weights for policy 0, policy_version 87320 (0.0009) -[2023-10-15 18:16:11,464][52866] Updated weights for policy 1, policy_version 87560 (0.0008) -[2023-10-15 18:16:11,819][52866] Updated weights for policy 1, policy_version 87570 (0.0007) -[2023-10-15 18:16:12,192][52866] Updated weights for policy 1, policy_version 87580 (0.0008) -[2023-10-15 18:16:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 179109888. Throughput: 0: 1793.0, 1: 1774.8. Samples: 44785856. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:16:13,442][51532] Avg episode reward: [(0, '70.800'), (1, '68.470')] -[2023-10-15 18:16:15,106][52833] Updated weights for policy 0, policy_version 87330 (0.0008) -[2023-10-15 18:16:15,469][52833] Updated weights for policy 0, policy_version 87340 (0.0011) -[2023-10-15 18:16:15,830][52833] Updated weights for policy 0, policy_version 87350 (0.0009) -[2023-10-15 18:16:15,948][52866] Updated weights for policy 1, policy_version 87590 (0.0008) -[2023-10-15 18:16:16,194][52833] Updated weights for policy 0, policy_version 87360 (0.0008) -[2023-10-15 18:16:16,313][52866] Updated weights for policy 1, policy_version 87600 (0.0009) -[2023-10-15 18:16:16,680][52866] Updated weights for policy 1, policy_version 87610 (0.0007) -[2023-10-15 18:16:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 179175424. Throughput: 0: 1802.0, 1: 1799.8. Samples: 44797166. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:16:18,441][51532] Avg episode reward: [(0, '68.910'), (1, '66.720')] -[2023-10-15 18:16:19,990][52833] Updated weights for policy 0, policy_version 87370 (0.0010) -[2023-10-15 18:16:20,352][52833] Updated weights for policy 0, policy_version 87380 (0.0007) -[2023-10-15 18:16:20,488][52866] Updated weights for policy 1, policy_version 87620 (0.0009) -[2023-10-15 18:16:20,721][52833] Updated weights for policy 0, policy_version 87390 (0.0007) -[2023-10-15 18:16:20,849][52866] Updated weights for policy 1, policy_version 87630 (0.0007) -[2023-10-15 18:16:21,210][52866] Updated weights for policy 1, policy_version 87640 (0.0010) -[2023-10-15 18:16:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 179240960. Throughput: 0: 1794.7, 1: 1773.6. Samples: 44818160. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-15 18:16:23,442][51532] Avg episode reward: [(0, '71.980'), (1, '66.240')] -[2023-10-15 18:16:24,463][52833] Updated weights for policy 0, policy_version 87400 (0.0008) -[2023-10-15 18:16:24,835][52833] Updated weights for policy 0, policy_version 87410 (0.0009) -[2023-10-15 18:16:24,929][52866] Updated weights for policy 1, policy_version 87650 (0.0009) -[2023-10-15 18:16:25,204][52833] Updated weights for policy 0, policy_version 87420 (0.0008) -[2023-10-15 18:16:25,296][52866] Updated weights for policy 1, policy_version 87660 (0.0009) -[2023-10-15 18:16:25,661][52866] Updated weights for policy 1, policy_version 87670 (0.0009) -[2023-10-15 18:16:26,034][52866] Updated weights for policy 1, policy_version 87680 (0.0010) -[2023-10-15 18:16:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 179306496. Throughput: 0: 1792.4, 1: 1778.5. Samples: 44840730. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:16:28,441][51532] Avg episode reward: [(0, '71.490'), (1, '65.520')] -[2023-10-15 18:16:29,083][52833] Updated weights for policy 0, policy_version 87430 (0.0008) -[2023-10-15 18:16:29,451][52833] Updated weights for policy 0, policy_version 87440 (0.0007) -[2023-10-15 18:16:29,579][52866] Updated weights for policy 1, policy_version 87690 (0.0009) -[2023-10-15 18:16:29,822][52833] Updated weights for policy 0, policy_version 87450 (0.0008) -[2023-10-15 18:16:29,947][52866] Updated weights for policy 1, policy_version 87700 (0.0008) -[2023-10-15 18:16:30,315][52866] Updated weights for policy 1, policy_version 87710 (0.0007) -[2023-10-15 18:16:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 179372032. Throughput: 0: 1790.9, 1: 1777.4. Samples: 44850380. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:16:33,442][51532] Avg episode reward: [(0, '70.660'), (1, '66.920')] -[2023-10-15 18:16:33,503][52833] Updated weights for policy 0, policy_version 87460 (0.0008) -[2023-10-15 18:16:33,868][52833] Updated weights for policy 0, policy_version 87470 (0.0009) -[2023-10-15 18:16:34,058][52866] Updated weights for policy 1, policy_version 87720 (0.0007) -[2023-10-15 18:16:34,235][52833] Updated weights for policy 0, policy_version 87480 (0.0007) -[2023-10-15 18:16:34,420][52866] Updated weights for policy 1, policy_version 87730 (0.0007) -[2023-10-15 18:16:34,784][52866] Updated weights for policy 1, policy_version 87740 (0.0009) -[2023-10-15 18:16:38,105][52833] Updated weights for policy 0, policy_version 87490 (0.0007) -[2023-10-15 18:16:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 179437568. Throughput: 0: 1787.1, 1: 1771.1. Samples: 44872606. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:16:38,442][51532] Avg episode reward: [(0, '71.780'), (1, '66.210')] -[2023-10-15 18:16:38,477][52833] Updated weights for policy 0, policy_version 87500 (0.0010) -[2023-10-15 18:16:38,576][52866] Updated weights for policy 1, policy_version 87750 (0.0009) -[2023-10-15 18:16:38,850][52833] Updated weights for policy 0, policy_version 87510 (0.0008) -[2023-10-15 18:16:38,945][52866] Updated weights for policy 1, policy_version 87760 (0.0008) -[2023-10-15 18:16:39,209][52833] Updated weights for policy 0, policy_version 87520 (0.0009) -[2023-10-15 18:16:39,313][52866] Updated weights for policy 1, policy_version 87770 (0.0007) -[2023-10-15 18:16:42,884][52833] Updated weights for policy 0, policy_version 87530 (0.0007) -[2023-10-15 18:16:43,012][52866] Updated weights for policy 1, policy_version 87780 (0.0007) -[2023-10-15 18:16:43,248][52833] Updated weights for policy 0, policy_version 87540 (0.0007) -[2023-10-15 18:16:43,379][52866] Updated weights for policy 1, policy_version 87790 (0.0007) -[2023-10-15 18:16:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 179503104. Throughput: 0: 1800.9, 1: 1796.8. Samples: 44894344. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:16:43,441][51532] Avg episode reward: [(0, '73.570'), (1, '67.140')] -[2023-10-15 18:16:43,614][52833] Updated weights for policy 0, policy_version 87550 (0.0007) -[2023-10-15 18:16:43,744][52866] Updated weights for policy 1, policy_version 87800 (0.0008) -[2023-10-15 18:16:47,280][52833] Updated weights for policy 0, policy_version 87560 (0.0010) -[2023-10-15 18:16:47,544][52866] Updated weights for policy 1, policy_version 87810 (0.0010) -[2023-10-15 18:16:47,645][52833] Updated weights for policy 0, policy_version 87570 (0.0008) -[2023-10-15 18:16:47,942][52866] Updated weights for policy 1, policy_version 87820 (0.0009) -[2023-10-15 18:16:48,013][52833] Updated weights for policy 0, policy_version 87580 (0.0008) -[2023-10-15 18:16:48,308][52866] Updated weights for policy 1, policy_version 87830 (0.0008) -[2023-10-15 18:16:48,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 179601408. Throughput: 0: 1788.7, 1: 1787.8. Samples: 44905094. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:16:48,441][51532] Avg episode reward: [(0, '74.150'), (1, '64.770')] -[2023-10-15 18:16:48,665][52866] Updated weights for policy 1, policy_version 87840 (0.0008) -[2023-10-15 18:16:51,755][52833] Updated weights for policy 0, policy_version 87590 (0.0007) -[2023-10-15 18:16:52,135][52833] Updated weights for policy 0, policy_version 87600 (0.0007) -[2023-10-15 18:16:52,473][52866] Updated weights for policy 1, policy_version 87850 (0.0007) -[2023-10-15 18:16:52,498][52833] Updated weights for policy 0, policy_version 87610 (0.0007) -[2023-10-15 18:16:52,843][52866] Updated weights for policy 1, policy_version 87860 (0.0009) -[2023-10-15 18:16:53,217][52866] Updated weights for policy 1, policy_version 87870 (0.0009) -[2023-10-15 18:16:53,441][51532] Fps is (10 sec: 19660.9, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 179699712. Throughput: 0: 1805.4, 1: 1803.4. Samples: 44926926. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:16:53,441][51532] Avg episode reward: [(0, '77.030'), (1, '63.930')] -[2023-10-15 18:16:56,340][52833] Updated weights for policy 0, policy_version 87620 (0.0008) -[2023-10-15 18:16:56,712][52833] Updated weights for policy 0, policy_version 87630 (0.0007) -[2023-10-15 18:16:57,071][52833] Updated weights for policy 0, policy_version 87640 (0.0008) -[2023-10-15 18:16:57,082][52866] Updated weights for policy 1, policy_version 87880 (0.0008) -[2023-10-15 18:16:57,455][52866] Updated weights for policy 1, policy_version 87890 (0.0007) -[2023-10-15 18:16:57,817][52866] Updated weights for policy 1, policy_version 87900 (0.0007) -[2023-10-15 18:16:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 179765248. Throughput: 0: 1784.7, 1: 1794.5. Samples: 44946920. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:16:58,441][51532] Avg episode reward: [(0, '76.470'), (1, '64.870')] -[2023-10-15 18:17:00,915][52833] Updated weights for policy 0, policy_version 87650 (0.0008) -[2023-10-15 18:17:01,273][52833] Updated weights for policy 0, policy_version 87660 (0.0010) -[2023-10-15 18:17:01,571][52866] Updated weights for policy 1, policy_version 87910 (0.0008) -[2023-10-15 18:17:01,642][52833] Updated weights for policy 0, policy_version 87670 (0.0007) -[2023-10-15 18:17:01,936][52866] Updated weights for policy 1, policy_version 87920 (0.0007) -[2023-10-15 18:17:02,007][52833] Updated weights for policy 0, policy_version 87680 (0.0007) -[2023-10-15 18:17:02,308][52866] Updated weights for policy 1, policy_version 87930 (0.0007) -[2023-10-15 18:17:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 179830784. Throughput: 0: 1806.1, 1: 1803.7. Samples: 44959610. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:17:03,441][51532] Avg episode reward: [(0, '75.900'), (1, '65.050')] -[2023-10-15 18:17:05,721][52833] Updated weights for policy 0, policy_version 87690 (0.0007) -[2023-10-15 18:17:05,878][52866] Updated weights for policy 1, policy_version 87940 (0.0007) -[2023-10-15 18:17:06,085][52833] Updated weights for policy 0, policy_version 87700 (0.0009) -[2023-10-15 18:17:06,238][52866] Updated weights for policy 1, policy_version 87950 (0.0008) -[2023-10-15 18:17:06,448][52833] Updated weights for policy 0, policy_version 87710 (0.0009) -[2023-10-15 18:17:06,606][52866] Updated weights for policy 1, policy_version 87960 (0.0008) -[2023-10-15 18:17:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 179896320. Throughput: 0: 1785.0, 1: 1804.1. Samples: 44979668. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:17:08,442][51532] Avg episode reward: [(0, '73.390'), (1, '63.100')] -[2023-10-15 18:17:10,228][52833] Updated weights for policy 0, policy_version 87720 (0.0009) -[2023-10-15 18:17:10,296][52866] Updated weights for policy 1, policy_version 87970 (0.0008) -[2023-10-15 18:17:10,603][52833] Updated weights for policy 0, policy_version 87730 (0.0008) -[2023-10-15 18:17:10,656][52866] Updated weights for policy 1, policy_version 87980 (0.0009) -[2023-10-15 18:17:10,976][52833] Updated weights for policy 0, policy_version 87740 (0.0009) -[2023-10-15 18:17:11,021][52866] Updated weights for policy 1, policy_version 87990 (0.0008) -[2023-10-15 18:17:11,389][52866] Updated weights for policy 1, policy_version 88000 (0.0010) -[2023-10-15 18:17:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 179961856. Throughput: 0: 1789.6, 1: 1801.9. Samples: 45002344. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:17:13,441][51532] Avg episode reward: [(0, '75.980'), (1, '64.340')] -[2023-10-15 18:17:14,599][52833] Updated weights for policy 0, policy_version 87750 (0.0008) -[2023-10-15 18:17:14,973][52833] Updated weights for policy 0, policy_version 87760 (0.0009) -[2023-10-15 18:17:15,087][52866] Updated weights for policy 1, policy_version 88010 (0.0007) -[2023-10-15 18:17:15,334][52833] Updated weights for policy 0, policy_version 87770 (0.0008) -[2023-10-15 18:17:15,453][52866] Updated weights for policy 1, policy_version 88020 (0.0007) -[2023-10-15 18:17:15,813][52866] Updated weights for policy 1, policy_version 88030 (0.0007) -[2023-10-15 18:17:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 180027392. Throughput: 0: 1792.2, 1: 1806.8. Samples: 45012336. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) -[2023-10-15 18:17:18,442][51532] Avg episode reward: [(0, '73.050'), (1, '62.830')] -[2023-10-15 18:17:19,041][52833] Updated weights for policy 0, policy_version 87780 (0.0009) -[2023-10-15 18:17:19,405][52833] Updated weights for policy 0, policy_version 87790 (0.0007) -[2023-10-15 18:17:19,478][52866] Updated weights for policy 1, policy_version 88040 (0.0008) -[2023-10-15 18:17:19,767][52833] Updated weights for policy 0, policy_version 87800 (0.0007) -[2023-10-15 18:17:19,839][52866] Updated weights for policy 1, policy_version 88050 (0.0007) -[2023-10-15 18:17:20,213][52866] Updated weights for policy 1, policy_version 88060 (0.0008) -[2023-10-15 18:17:23,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 180092928. Throughput: 0: 1792.3, 1: 1815.9. Samples: 45034976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:17:23,442][51532] Avg episode reward: [(0, '72.010'), (1, '62.490')] -[2023-10-15 18:17:23,573][52833] Updated weights for policy 0, policy_version 87810 (0.0008) -[2023-10-15 18:17:23,953][52833] Updated weights for policy 0, policy_version 87820 (0.0010) -[2023-10-15 18:17:23,968][52866] Updated weights for policy 1, policy_version 88070 (0.0008) -[2023-10-15 18:17:24,311][52833] Updated weights for policy 0, policy_version 87830 (0.0010) -[2023-10-15 18:17:24,337][52866] Updated weights for policy 1, policy_version 88080 (0.0007) -[2023-10-15 18:17:24,675][52833] Updated weights for policy 0, policy_version 87840 (0.0010) -[2023-10-15 18:17:24,705][52866] Updated weights for policy 1, policy_version 88090 (0.0007) -[2023-10-15 18:17:28,428][52866] Updated weights for policy 1, policy_version 88100 (0.0010) -[2023-10-15 18:17:28,436][52833] Updated weights for policy 0, policy_version 87850 (0.0007) -[2023-10-15 18:17:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 180158464. Throughput: 0: 1807.1, 1: 1815.2. Samples: 45057346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:17:28,441][51532] Avg episode reward: [(0, '72.360'), (1, '61.940')] -[2023-10-15 18:17:28,794][52866] Updated weights for policy 1, policy_version 88110 (0.0008) -[2023-10-15 18:17:28,804][52833] Updated weights for policy 0, policy_version 87860 (0.0009) -[2023-10-15 18:17:29,162][52866] Updated weights for policy 1, policy_version 88120 (0.0009) -[2023-10-15 18:17:29,165][52833] Updated weights for policy 0, policy_version 87870 (0.0009) -[2023-10-15 18:17:32,875][52833] Updated weights for policy 0, policy_version 87880 (0.0008) -[2023-10-15 18:17:32,949][52866] Updated weights for policy 1, policy_version 88130 (0.0009) -[2023-10-15 18:17:33,249][52833] Updated weights for policy 0, policy_version 87890 (0.0007) -[2023-10-15 18:17:33,339][52866] Updated weights for policy 1, policy_version 88140 (0.0007) -[2023-10-15 18:17:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 180224000. Throughput: 0: 1793.8, 1: 1810.7. Samples: 45067294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:17:33,442][51532] Avg episode reward: [(0, '74.120'), (1, '62.580')] -[2023-10-15 18:17:33,632][52833] Updated weights for policy 0, policy_version 87900 (0.0007) -[2023-10-15 18:17:33,703][52866] Updated weights for policy 1, policy_version 88150 (0.0008) -[2023-10-15 18:17:34,066][52866] Updated weights for policy 1, policy_version 88160 (0.0009) -[2023-10-15 18:17:37,301][52833] Updated weights for policy 0, policy_version 87910 (0.0010) -[2023-10-15 18:17:37,671][52833] Updated weights for policy 0, policy_version 87920 (0.0008) -[2023-10-15 18:17:37,733][52866] Updated weights for policy 1, policy_version 88170 (0.0008) -[2023-10-15 18:17:38,035][52833] Updated weights for policy 0, policy_version 87930 (0.0008) -[2023-10-15 18:17:38,106][52866] Updated weights for policy 1, policy_version 88180 (0.0007) -[2023-10-15 18:17:38,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 180322304. Throughput: 0: 1805.0, 1: 1810.3. Samples: 45089616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:17:38,442][51532] Avg episode reward: [(0, '72.800'), (1, '61.370')] -[2023-10-15 18:17:38,467][52866] Updated weights for policy 1, policy_version 88190 (0.0008) -[2023-10-15 18:17:41,779][52833] Updated weights for policy 0, policy_version 87940 (0.0008) -[2023-10-15 18:17:42,154][52833] Updated weights for policy 0, policy_version 87950 (0.0008) -[2023-10-15 18:17:42,227][52866] Updated weights for policy 1, policy_version 88200 (0.0009) -[2023-10-15 18:17:42,525][52833] Updated weights for policy 0, policy_version 87960 (0.0009) -[2023-10-15 18:17:42,591][52866] Updated weights for policy 1, policy_version 88210 (0.0008) -[2023-10-15 18:17:42,967][52866] Updated weights for policy 1, policy_version 88220 (0.0007) -[2023-10-15 18:17:43,441][51532] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 180420608. Throughput: 0: 1798.6, 1: 1816.8. Samples: 45109612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:17:43,441][51532] Avg episode reward: [(0, '73.140'), (1, '61.570')] -[2023-10-15 18:17:43,450][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000088224_90341376.pth... -[2023-10-15 18:17:43,450][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000087968_90079232.pth... -[2023-10-15 18:17:43,486][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000086528_88604672.pth -[2023-10-15 18:17:43,486][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000086272_88342528.pth -[2023-10-15 18:17:46,189][52833] Updated weights for policy 0, policy_version 87970 (0.0009) -[2023-10-15 18:17:46,574][52833] Updated weights for policy 0, policy_version 87980 (0.0008) -[2023-10-15 18:17:46,624][52866] Updated weights for policy 1, policy_version 88230 (0.0008) -[2023-10-15 18:17:46,938][52833] Updated weights for policy 0, policy_version 87990 (0.0007) -[2023-10-15 18:17:46,994][52866] Updated weights for policy 1, policy_version 88240 (0.0007) -[2023-10-15 18:17:47,301][52833] Updated weights for policy 0, policy_version 88000 (0.0007) -[2023-10-15 18:17:47,354][52866] Updated weights for policy 1, policy_version 88250 (0.0007) -[2023-10-15 18:17:48,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 180486144. Throughput: 0: 1800.1, 1: 1812.7. Samples: 45122186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:17:48,441][51532] Avg episode reward: [(0, '70.040'), (1, '62.540')] -[2023-10-15 18:17:51,104][52833] Updated weights for policy 0, policy_version 88010 (0.0008) -[2023-10-15 18:17:51,152][52866] Updated weights for policy 1, policy_version 88260 (0.0009) -[2023-10-15 18:17:51,473][52833] Updated weights for policy 0, policy_version 88020 (0.0008) -[2023-10-15 18:17:51,520][52866] Updated weights for policy 1, policy_version 88270 (0.0008) -[2023-10-15 18:17:51,840][52833] Updated weights for policy 0, policy_version 88030 (0.0007) -[2023-10-15 18:17:51,881][52866] Updated weights for policy 1, policy_version 88280 (0.0009) -[2023-10-15 18:17:53,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 180551680. Throughput: 0: 1794.8, 1: 1811.7. Samples: 45141962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:17:53,442][51532] Avg episode reward: [(0, '70.050'), (1, '65.020')] -[2023-10-15 18:17:55,592][52866] Updated weights for policy 1, policy_version 88290 (0.0010) -[2023-10-15 18:17:55,673][52833] Updated weights for policy 0, policy_version 88040 (0.0008) -[2023-10-15 18:17:55,967][52866] Updated weights for policy 1, policy_version 88300 (0.0008) -[2023-10-15 18:17:56,038][52833] Updated weights for policy 0, policy_version 88050 (0.0007) -[2023-10-15 18:17:56,326][52866] Updated weights for policy 1, policy_version 88310 (0.0008) -[2023-10-15 18:17:56,412][52833] Updated weights for policy 0, policy_version 88060 (0.0009) -[2023-10-15 18:17:56,693][52866] Updated weights for policy 1, policy_version 88320 (0.0010) -[2023-10-15 18:17:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 180617216. Throughput: 0: 1791.5, 1: 1804.8. Samples: 45164178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:17:58,442][51532] Avg episode reward: [(0, '71.590'), (1, '65.520')] -[2023-10-15 18:18:00,232][52833] Updated weights for policy 0, policy_version 88070 (0.0008) -[2023-10-15 18:18:00,411][52866] Updated weights for policy 1, policy_version 88330 (0.0007) -[2023-10-15 18:18:00,606][52833] Updated weights for policy 0, policy_version 88080 (0.0008) -[2023-10-15 18:18:00,777][52866] Updated weights for policy 1, policy_version 88340 (0.0008) -[2023-10-15 18:18:00,966][52833] Updated weights for policy 0, policy_version 88090 (0.0007) -[2023-10-15 18:18:01,146][52866] Updated weights for policy 1, policy_version 88350 (0.0007) -[2023-10-15 18:18:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 180682752. Throughput: 0: 1798.5, 1: 1813.5. Samples: 45174878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:03,442][51532] Avg episode reward: [(0, '73.830'), (1, '64.790')] -[2023-10-15 18:18:04,655][52833] Updated weights for policy 0, policy_version 88100 (0.0010) -[2023-10-15 18:18:04,662][52866] Updated weights for policy 1, policy_version 88360 (0.0008) -[2023-10-15 18:18:05,027][52866] Updated weights for policy 1, policy_version 88370 (0.0007) -[2023-10-15 18:18:05,029][52833] Updated weights for policy 0, policy_version 88110 (0.0008) -[2023-10-15 18:18:05,381][52866] Updated weights for policy 1, policy_version 88380 (0.0008) -[2023-10-15 18:18:05,392][52833] Updated weights for policy 0, policy_version 88120 (0.0007) -[2023-10-15 18:18:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 180748288. Throughput: 0: 1791.2, 1: 1803.4. Samples: 45196736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:08,442][51532] Avg episode reward: [(0, '76.590'), (1, '68.430')] -[2023-10-15 18:18:09,050][52833] Updated weights for policy 0, policy_version 88130 (0.0008) -[2023-10-15 18:18:09,135][52866] Updated weights for policy 1, policy_version 88390 (0.0008) -[2023-10-15 18:18:09,413][52833] Updated weights for policy 0, policy_version 88140 (0.0007) -[2023-10-15 18:18:09,503][52866] Updated weights for policy 1, policy_version 88400 (0.0008) -[2023-10-15 18:18:09,785][52833] Updated weights for policy 0, policy_version 88150 (0.0008) -[2023-10-15 18:18:09,867][52866] Updated weights for policy 1, policy_version 88410 (0.0007) -[2023-10-15 18:18:10,158][52833] Updated weights for policy 0, policy_version 88160 (0.0009) -[2023-10-15 18:18:13,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 180813824. Throughput: 0: 1798.4, 1: 1807.9. Samples: 45219630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:13,442][51532] Avg episode reward: [(0, '74.150'), (1, '73.610')] -[2023-10-15 18:18:13,547][52866] Updated weights for policy 1, policy_version 88420 (0.0007) -[2023-10-15 18:18:13,916][52866] Updated weights for policy 1, policy_version 88430 (0.0008) -[2023-10-15 18:18:13,957][52833] Updated weights for policy 0, policy_version 88170 (0.0007) -[2023-10-15 18:18:14,282][52866] Updated weights for policy 1, policy_version 88440 (0.0008) -[2023-10-15 18:18:14,329][52833] Updated weights for policy 0, policy_version 88180 (0.0007) -[2023-10-15 18:18:14,707][52833] Updated weights for policy 0, policy_version 88190 (0.0007) -[2023-10-15 18:18:18,135][52866] Updated weights for policy 1, policy_version 88450 (0.0007) -[2023-10-15 18:18:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 180879360. Throughput: 0: 1792.9, 1: 1807.8. Samples: 45229324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:18,442][51532] Avg episode reward: [(0, '74.870'), (1, '70.130')] -[2023-10-15 18:18:18,521][52866] Updated weights for policy 1, policy_version 88460 (0.0008) -[2023-10-15 18:18:18,550][52833] Updated weights for policy 0, policy_version 88200 (0.0008) -[2023-10-15 18:18:18,893][52866] Updated weights for policy 1, policy_version 88470 (0.0007) -[2023-10-15 18:18:18,925][52833] Updated weights for policy 0, policy_version 88210 (0.0008) -[2023-10-15 18:18:19,252][52866] Updated weights for policy 1, policy_version 88480 (0.0008) -[2023-10-15 18:18:19,283][52833] Updated weights for policy 0, policy_version 88220 (0.0007) -[2023-10-15 18:18:23,084][52866] Updated weights for policy 1, policy_version 88490 (0.0008) -[2023-10-15 18:18:23,088][52833] Updated weights for policy 0, policy_version 88230 (0.0008) -[2023-10-15 18:18:23,440][52866] Updated weights for policy 1, policy_version 88500 (0.0007) -[2023-10-15 18:18:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14218.0). Total num frames: 180944896. Throughput: 0: 1792.8, 1: 1802.5. Samples: 45251406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:23,442][51532] Avg episode reward: [(0, '79.500'), (1, '69.300')] -[2023-10-15 18:18:23,447][52833] Updated weights for policy 0, policy_version 88240 (0.0007) -[2023-10-15 18:18:23,806][52866] Updated weights for policy 1, policy_version 88510 (0.0008) -[2023-10-15 18:18:23,818][52833] Updated weights for policy 0, policy_version 88250 (0.0008) -[2023-10-15 18:18:27,492][52833] Updated weights for policy 0, policy_version 88260 (0.0008) -[2023-10-15 18:18:27,547][52866] Updated weights for policy 1, policy_version 88520 (0.0008) -[2023-10-15 18:18:27,855][52833] Updated weights for policy 0, policy_version 88270 (0.0009) -[2023-10-15 18:18:27,916][52866] Updated weights for policy 1, policy_version 88530 (0.0007) -[2023-10-15 18:18:28,212][52833] Updated weights for policy 0, policy_version 88280 (0.0008) -[2023-10-15 18:18:28,282][52866] Updated weights for policy 1, policy_version 88540 (0.0008) -[2023-10-15 18:18:28,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 181043200. Throughput: 0: 1806.7, 1: 1814.3. Samples: 45272556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:28,441][51532] Avg episode reward: [(0, '80.530'), (1, '69.270')] -[2023-10-15 18:18:31,948][52833] Updated weights for policy 0, policy_version 88290 (0.0008) -[2023-10-15 18:18:31,962][52866] Updated weights for policy 1, policy_version 88550 (0.0008) -[2023-10-15 18:18:32,317][52833] Updated weights for policy 0, policy_version 88300 (0.0007) -[2023-10-15 18:18:32,329][52866] Updated weights for policy 1, policy_version 88560 (0.0008) -[2023-10-15 18:18:32,695][52833] Updated weights for policy 0, policy_version 88310 (0.0007) -[2023-10-15 18:18:32,697][52866] Updated weights for policy 1, policy_version 88570 (0.0007) -[2023-10-15 18:18:33,053][52833] Updated weights for policy 0, policy_version 88320 (0.0009) -[2023-10-15 18:18:33,441][51532] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 181141504. Throughput: 0: 1791.2, 1: 1806.6. Samples: 45284086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:33,441][51532] Avg episode reward: [(0, '82.580'), (1, '68.150')] -[2023-10-15 18:18:36,269][52866] Updated weights for policy 1, policy_version 88580 (0.0007) -[2023-10-15 18:18:36,633][52866] Updated weights for policy 1, policy_version 88590 (0.0008) -[2023-10-15 18:18:36,823][52833] Updated weights for policy 0, policy_version 88330 (0.0008) -[2023-10-15 18:18:36,991][52866] Updated weights for policy 1, policy_version 88600 (0.0007) -[2023-10-15 18:18:37,180][52833] Updated weights for policy 0, policy_version 88340 (0.0007) -[2023-10-15 18:18:37,560][52833] Updated weights for policy 0, policy_version 88350 (0.0009) -[2023-10-15 18:18:38,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 181207040. Throughput: 0: 1814.0, 1: 1813.3. Samples: 45305192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:38,442][51532] Avg episode reward: [(0, '86.440'), (1, '67.110')] -[2023-10-15 18:18:40,649][52866] Updated weights for policy 1, policy_version 88610 (0.0008) -[2023-10-15 18:18:41,017][52866] Updated weights for policy 1, policy_version 88620 (0.0008) -[2023-10-15 18:18:41,265][52833] Updated weights for policy 0, policy_version 88360 (0.0009) -[2023-10-15 18:18:41,380][52866] Updated weights for policy 1, policy_version 88630 (0.0009) -[2023-10-15 18:18:41,628][52833] Updated weights for policy 0, policy_version 88370 (0.0009) -[2023-10-15 18:18:41,747][52866] Updated weights for policy 1, policy_version 88640 (0.0009) -[2023-10-15 18:18:42,007][52833] Updated weights for policy 0, policy_version 88380 (0.0008) -[2023-10-15 18:18:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 181272576. Throughput: 0: 1796.7, 1: 1810.8. Samples: 45326514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:43,442][51532] Avg episode reward: [(0, '85.490'), (1, '69.110')] -[2023-10-15 18:18:45,546][52866] Updated weights for policy 1, policy_version 88650 (0.0008) -[2023-10-15 18:18:45,565][52833] Updated weights for policy 0, policy_version 88390 (0.0008) -[2023-10-15 18:18:45,910][52866] Updated weights for policy 1, policy_version 88660 (0.0007) -[2023-10-15 18:18:45,936][52833] Updated weights for policy 0, policy_version 88400 (0.0007) -[2023-10-15 18:18:46,273][52866] Updated weights for policy 1, policy_version 88670 (0.0010) -[2023-10-15 18:18:46,310][52833] Updated weights for policy 0, policy_version 88410 (0.0008) -[2023-10-15 18:18:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 181338112. Throughput: 0: 1808.1, 1: 1809.3. Samples: 45337662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:48,441][51532] Avg episode reward: [(0, '84.750'), (1, '69.220')] -[2023-10-15 18:18:50,069][52833] Updated weights for policy 0, policy_version 88420 (0.0008) -[2023-10-15 18:18:50,071][52866] Updated weights for policy 1, policy_version 88680 (0.0009) -[2023-10-15 18:18:50,430][52833] Updated weights for policy 0, policy_version 88430 (0.0008) -[2023-10-15 18:18:50,434][52866] Updated weights for policy 1, policy_version 88690 (0.0008) -[2023-10-15 18:18:50,798][52866] Updated weights for policy 1, policy_version 88700 (0.0009) -[2023-10-15 18:18:50,801][52833] Updated weights for policy 0, policy_version 88440 (0.0008) -[2023-10-15 18:18:53,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 181403648. Throughput: 0: 1795.6, 1: 1800.7. Samples: 45358568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:53,442][51532] Avg episode reward: [(0, '86.420'), (1, '70.210')] -[2023-10-15 18:18:54,580][52833] Updated weights for policy 0, policy_version 88450 (0.0008) -[2023-10-15 18:18:54,713][52866] Updated weights for policy 1, policy_version 88710 (0.0008) -[2023-10-15 18:18:54,957][52833] Updated weights for policy 0, policy_version 88460 (0.0008) -[2023-10-15 18:18:55,078][52866] Updated weights for policy 1, policy_version 88720 (0.0009) -[2023-10-15 18:18:55,326][52833] Updated weights for policy 0, policy_version 88470 (0.0008) -[2023-10-15 18:18:55,445][52866] Updated weights for policy 1, policy_version 88730 (0.0009) -[2023-10-15 18:18:55,700][52833] Updated weights for policy 0, policy_version 88480 (0.0007) -[2023-10-15 18:18:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 181469184. Throughput: 0: 1785.4, 1: 1796.6. Samples: 45380820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:18:58,442][51532] Avg episode reward: [(0, '84.690'), (1, '67.340')] -[2023-10-15 18:18:58,967][52866] Updated weights for policy 1, policy_version 88740 (0.0009) -[2023-10-15 18:18:59,339][52866] Updated weights for policy 1, policy_version 88750 (0.0009) -[2023-10-15 18:18:59,554][52833] Updated weights for policy 0, policy_version 88490 (0.0008) -[2023-10-15 18:18:59,696][52866] Updated weights for policy 1, policy_version 88760 (0.0009) -[2023-10-15 18:18:59,922][52833] Updated weights for policy 0, policy_version 88500 (0.0007) -[2023-10-15 18:19:00,300][52833] Updated weights for policy 0, policy_version 88510 (0.0010) -[2023-10-15 18:19:03,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 181534720. Throughput: 0: 1787.7, 1: 1795.3. Samples: 45390558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:19:03,441][51532] Avg episode reward: [(0, '83.230'), (1, '67.840')] -[2023-10-15 18:19:03,501][52866] Updated weights for policy 1, policy_version 88770 (0.0009) -[2023-10-15 18:19:03,907][52866] Updated weights for policy 1, policy_version 88780 (0.0009) -[2023-10-15 18:19:04,067][52833] Updated weights for policy 0, policy_version 88520 (0.0009) -[2023-10-15 18:19:04,266][52866] Updated weights for policy 1, policy_version 88790 (0.0007) -[2023-10-15 18:19:04,429][52833] Updated weights for policy 0, policy_version 88530 (0.0010) -[2023-10-15 18:19:04,621][52866] Updated weights for policy 1, policy_version 88800 (0.0007) -[2023-10-15 18:19:04,799][52833] Updated weights for policy 0, policy_version 88540 (0.0009) -[2023-10-15 18:19:08,337][52866] Updated weights for policy 1, policy_version 88810 (0.0007) -[2023-10-15 18:19:08,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 181600256. Throughput: 0: 1789.7, 1: 1799.0. Samples: 45412896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:19:08,441][51532] Avg episode reward: [(0, '81.970'), (1, '63.380')] -[2023-10-15 18:19:08,459][52833] Updated weights for policy 0, policy_version 88550 (0.0008) -[2023-10-15 18:19:08,697][52866] Updated weights for policy 1, policy_version 88820 (0.0007) -[2023-10-15 18:19:08,820][52833] Updated weights for policy 0, policy_version 88560 (0.0008) -[2023-10-15 18:19:09,067][52866] Updated weights for policy 1, policy_version 88830 (0.0008) -[2023-10-15 18:19:09,189][52833] Updated weights for policy 0, policy_version 88570 (0.0007) -[2023-10-15 18:19:12,950][52866] Updated weights for policy 1, policy_version 88840 (0.0008) -[2023-10-15 18:19:12,987][52833] Updated weights for policy 0, policy_version 88580 (0.0010) -[2023-10-15 18:19:13,319][52866] Updated weights for policy 1, policy_version 88850 (0.0008) -[2023-10-15 18:19:13,348][52833] Updated weights for policy 0, policy_version 88590 (0.0009) -[2023-10-15 18:19:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 181665792. Throughput: 0: 1807.5, 1: 1803.0. Samples: 45435028. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:13,442][51532] Avg episode reward: [(0, '81.700'), (1, '62.980')] -[2023-10-15 18:19:13,691][52866] Updated weights for policy 1, policy_version 88860 (0.0008) -[2023-10-15 18:19:13,718][52833] Updated weights for policy 0, policy_version 88600 (0.0009) -[2023-10-15 18:19:17,477][52833] Updated weights for policy 0, policy_version 88610 (0.0008) -[2023-10-15 18:19:17,632][52866] Updated weights for policy 1, policy_version 88870 (0.0009) -[2023-10-15 18:19:17,854][52833] Updated weights for policy 0, policy_version 88620 (0.0008) -[2023-10-15 18:19:17,989][52866] Updated weights for policy 1, policy_version 88880 (0.0009) -[2023-10-15 18:19:18,224][52833] Updated weights for policy 0, policy_version 88630 (0.0007) -[2023-10-15 18:19:18,362][52866] Updated weights for policy 1, policy_version 88890 (0.0008) -[2023-10-15 18:19:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 181731328. Throughput: 0: 1789.4, 1: 1788.3. Samples: 45445082. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:18,442][51532] Avg episode reward: [(0, '82.780'), (1, '64.130')] -[2023-10-15 18:19:18,586][52833] Updated weights for policy 0, policy_version 88640 (0.0008) -[2023-10-15 18:19:22,083][52866] Updated weights for policy 1, policy_version 88900 (0.0009) -[2023-10-15 18:19:22,264][52833] Updated weights for policy 0, policy_version 88650 (0.0007) -[2023-10-15 18:19:22,444][52866] Updated weights for policy 1, policy_version 88910 (0.0008) -[2023-10-15 18:19:22,630][52833] Updated weights for policy 0, policy_version 88660 (0.0008) -[2023-10-15 18:19:22,809][52866] Updated weights for policy 1, policy_version 88920 (0.0007) -[2023-10-15 18:19:22,993][52833] Updated weights for policy 0, policy_version 88670 (0.0008) -[2023-10-15 18:19:23,441][51532] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 181862400. Throughput: 0: 1801.8, 1: 1808.0. Samples: 45467636. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:23,441][51532] Avg episode reward: [(0, '84.110'), (1, '63.660')] -[2023-10-15 18:19:26,580][52833] Updated weights for policy 0, policy_version 88680 (0.0008) -[2023-10-15 18:19:26,693][52866] Updated weights for policy 1, policy_version 88930 (0.0008) -[2023-10-15 18:19:26,940][52833] Updated weights for policy 0, policy_version 88690 (0.0008) -[2023-10-15 18:19:27,046][52866] Updated weights for policy 1, policy_version 88940 (0.0008) -[2023-10-15 18:19:27,313][52833] Updated weights for policy 0, policy_version 88700 (0.0008) -[2023-10-15 18:19:27,419][52866] Updated weights for policy 1, policy_version 88950 (0.0007) -[2023-10-15 18:19:27,786][52866] Updated weights for policy 1, policy_version 88960 (0.0007) -[2023-10-15 18:19:28,441][51532] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 181927936. Throughput: 0: 1791.2, 1: 1777.4. Samples: 45487102. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:28,442][51532] Avg episode reward: [(0, '80.700'), (1, '65.210')] -[2023-10-15 18:19:31,069][52833] Updated weights for policy 0, policy_version 88710 (0.0008) -[2023-10-15 18:19:31,436][52833] Updated weights for policy 0, policy_version 88720 (0.0010) -[2023-10-15 18:19:31,560][52866] Updated weights for policy 1, policy_version 88970 (0.0009) -[2023-10-15 18:19:31,803][52833] Updated weights for policy 0, policy_version 88730 (0.0007) -[2023-10-15 18:19:31,920][52866] Updated weights for policy 1, policy_version 88980 (0.0007) -[2023-10-15 18:19:32,283][52866] Updated weights for policy 1, policy_version 88990 (0.0008) -[2023-10-15 18:19:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 181993472. Throughput: 0: 1803.2, 1: 1800.2. Samples: 45499814. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:33,442][51532] Avg episode reward: [(0, '81.700'), (1, '65.430')] -[2023-10-15 18:19:35,738][52833] Updated weights for policy 0, policy_version 88740 (0.0008) -[2023-10-15 18:19:35,945][52866] Updated weights for policy 1, policy_version 89000 (0.0008) -[2023-10-15 18:19:36,104][52833] Updated weights for policy 0, policy_version 88750 (0.0009) -[2023-10-15 18:19:36,302][52866] Updated weights for policy 1, policy_version 89010 (0.0008) -[2023-10-15 18:19:36,476][52833] Updated weights for policy 0, policy_version 88760 (0.0008) -[2023-10-15 18:19:36,664][52866] Updated weights for policy 1, policy_version 89020 (0.0007) -[2023-10-15 18:19:38,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182059008. Throughput: 0: 1785.4, 1: 1780.3. Samples: 45519026. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:38,441][51532] Avg episode reward: [(0, '80.770'), (1, '64.290')] -[2023-10-15 18:19:40,401][52833] Updated weights for policy 0, policy_version 88770 (0.0007) -[2023-10-15 18:19:40,407][52866] Updated weights for policy 1, policy_version 89030 (0.0008) -[2023-10-15 18:19:40,764][52866] Updated weights for policy 1, policy_version 89040 (0.0009) -[2023-10-15 18:19:40,766][52833] Updated weights for policy 0, policy_version 88780 (0.0007) -[2023-10-15 18:19:41,124][52866] Updated weights for policy 1, policy_version 89050 (0.0008) -[2023-10-15 18:19:41,134][52833] Updated weights for policy 0, policy_version 88790 (0.0007) -[2023-10-15 18:19:41,501][52833] Updated weights for policy 0, policy_version 88800 (0.0009) -[2023-10-15 18:19:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 182124544. Throughput: 0: 1793.7, 1: 1778.5. Samples: 45541570. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:43,442][51532] Avg episode reward: [(0, '81.030'), (1, '63.680')] -[2023-10-15 18:19:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000088800_90931200.pth... -[2023-10-15 18:19:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000089056_91193344.pth... -[2023-10-15 18:19:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000087392_89489408.pth -[2023-10-15 18:19:43,495][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000087136_89227264.pth -[2023-10-15 18:19:44,921][52866] Updated weights for policy 1, policy_version 89060 (0.0008) -[2023-10-15 18:19:45,230][52833] Updated weights for policy 0, policy_version 88810 (0.0008) -[2023-10-15 18:19:45,285][52866] Updated weights for policy 1, policy_version 89070 (0.0008) -[2023-10-15 18:19:45,601][52833] Updated weights for policy 0, policy_version 88820 (0.0008) -[2023-10-15 18:19:45,649][52866] Updated weights for policy 1, policy_version 89080 (0.0010) -[2023-10-15 18:19:45,969][52833] Updated weights for policy 0, policy_version 88830 (0.0007) -[2023-10-15 18:19:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182190080. Throughput: 0: 1803.0, 1: 1779.2. Samples: 45551758. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:48,441][51532] Avg episode reward: [(0, '81.660'), (1, '64.530')] -[2023-10-15 18:19:49,455][52866] Updated weights for policy 1, policy_version 89090 (0.0007) -[2023-10-15 18:19:49,773][52833] Updated weights for policy 0, policy_version 88840 (0.0008) -[2023-10-15 18:19:49,828][52866] Updated weights for policy 1, policy_version 89100 (0.0007) -[2023-10-15 18:19:50,147][52833] Updated weights for policy 0, policy_version 88850 (0.0008) -[2023-10-15 18:19:50,189][52866] Updated weights for policy 1, policy_version 89110 (0.0008) -[2023-10-15 18:19:50,507][52833] Updated weights for policy 0, policy_version 88860 (0.0010) -[2023-10-15 18:19:50,553][52866] Updated weights for policy 1, policy_version 89120 (0.0009) -[2023-10-15 18:19:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 182255616. Throughput: 0: 1790.9, 1: 1779.3. Samples: 45573558. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:53,442][51532] Avg episode reward: [(0, '81.430'), (1, '62.900')] -[2023-10-15 18:19:54,370][52866] Updated weights for policy 1, policy_version 89130 (0.0008) -[2023-10-15 18:19:54,385][52833] Updated weights for policy 0, policy_version 88870 (0.0009) -[2023-10-15 18:19:54,740][52866] Updated weights for policy 1, policy_version 89140 (0.0008) -[2023-10-15 18:19:54,744][52833] Updated weights for policy 0, policy_version 88880 (0.0008) -[2023-10-15 18:19:55,107][52866] Updated weights for policy 1, policy_version 89150 (0.0009) -[2023-10-15 18:19:55,120][52833] Updated weights for policy 0, policy_version 88890 (0.0009) -[2023-10-15 18:19:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 182321152. Throughput: 0: 1788.4, 1: 1784.9. Samples: 45595826. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:19:58,442][51532] Avg episode reward: [(0, '80.320'), (1, '65.260')] -[2023-10-15 18:19:58,817][52866] Updated weights for policy 1, policy_version 89160 (0.0009) -[2023-10-15 18:19:58,887][52833] Updated weights for policy 0, policy_version 88900 (0.0007) -[2023-10-15 18:19:59,183][52866] Updated weights for policy 1, policy_version 89170 (0.0009) -[2023-10-15 18:19:59,254][52833] Updated weights for policy 0, policy_version 88910 (0.0007) -[2023-10-15 18:19:59,550][52866] Updated weights for policy 1, policy_version 89180 (0.0007) -[2023-10-15 18:19:59,622][52833] Updated weights for policy 0, policy_version 88920 (0.0007) -[2023-10-15 18:20:03,282][52866] Updated weights for policy 1, policy_version 89190 (0.0008) -[2023-10-15 18:20:03,334][52833] Updated weights for policy 0, policy_version 88930 (0.0007) -[2023-10-15 18:20:03,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 182386688. Throughput: 0: 1788.1, 1: 1780.7. Samples: 45605678. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) -[2023-10-15 18:20:03,441][51532] Avg episode reward: [(0, '79.490'), (1, '65.270')] -[2023-10-15 18:20:03,642][52866] Updated weights for policy 1, policy_version 89200 (0.0010) -[2023-10-15 18:20:03,702][52833] Updated weights for policy 0, policy_version 88940 (0.0007) -[2023-10-15 18:20:04,014][52866] Updated weights for policy 1, policy_version 89210 (0.0009) -[2023-10-15 18:20:04,071][52833] Updated weights for policy 0, policy_version 88950 (0.0008) -[2023-10-15 18:20:04,443][52833] Updated weights for policy 0, policy_version 88960 (0.0007) -[2023-10-15 18:20:07,744][52866] Updated weights for policy 1, policy_version 89220 (0.0009) -[2023-10-15 18:20:08,106][52866] Updated weights for policy 1, policy_version 89230 (0.0008) -[2023-10-15 18:20:08,179][52833] Updated weights for policy 0, policy_version 88970 (0.0008) -[2023-10-15 18:20:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 182452224. Throughput: 0: 1782.9, 1: 1780.9. Samples: 45628008. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:08,441][51532] Avg episode reward: [(0, '82.910'), (1, '66.350')] -[2023-10-15 18:20:08,474][52866] Updated weights for policy 1, policy_version 89240 (0.0007) -[2023-10-15 18:20:08,535][52833] Updated weights for policy 0, policy_version 88980 (0.0007) -[2023-10-15 18:20:08,909][52833] Updated weights for policy 0, policy_version 88990 (0.0009) -[2023-10-15 18:20:12,307][52866] Updated weights for policy 1, policy_version 89250 (0.0008) -[2023-10-15 18:20:12,644][52833] Updated weights for policy 0, policy_version 89000 (0.0007) -[2023-10-15 18:20:12,670][52866] Updated weights for policy 1, policy_version 89260 (0.0007) -[2023-10-15 18:20:13,008][52833] Updated weights for policy 0, policy_version 89010 (0.0008) -[2023-10-15 18:20:13,039][52866] Updated weights for policy 1, policy_version 89270 (0.0008) -[2023-10-15 18:20:13,375][52833] Updated weights for policy 0, policy_version 89020 (0.0007) -[2023-10-15 18:20:13,402][52866] Updated weights for policy 1, policy_version 89280 (0.0008) -[2023-10-15 18:20:13,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 182550528. Throughput: 0: 1802.7, 1: 1796.3. Samples: 45649058. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:13,441][51532] Avg episode reward: [(0, '83.360'), (1, '66.810')] -[2023-10-15 18:20:16,941][52866] Updated weights for policy 1, policy_version 89290 (0.0007) -[2023-10-15 18:20:17,157][52833] Updated weights for policy 0, policy_version 89030 (0.0009) -[2023-10-15 18:20:17,308][52866] Updated weights for policy 1, policy_version 89300 (0.0007) -[2023-10-15 18:20:17,522][52833] Updated weights for policy 0, policy_version 89040 (0.0009) -[2023-10-15 18:20:17,671][52866] Updated weights for policy 1, policy_version 89310 (0.0007) -[2023-10-15 18:20:17,887][52833] Updated weights for policy 0, policy_version 89050 (0.0009) -[2023-10-15 18:20:18,441][51532] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 182648832. Throughput: 0: 1786.0, 1: 1788.9. Samples: 45660682. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:18,441][51532] Avg episode reward: [(0, '81.220'), (1, '69.270')] -[2023-10-15 18:20:21,424][52866] Updated weights for policy 1, policy_version 89320 (0.0008) -[2023-10-15 18:20:21,688][52833] Updated weights for policy 0, policy_version 89060 (0.0009) -[2023-10-15 18:20:21,798][52866] Updated weights for policy 1, policy_version 89330 (0.0010) -[2023-10-15 18:20:22,059][52833] Updated weights for policy 0, policy_version 89070 (0.0010) -[2023-10-15 18:20:22,162][52866] Updated weights for policy 1, policy_version 89340 (0.0010) -[2023-10-15 18:20:22,439][52833] Updated weights for policy 0, policy_version 89080 (0.0009) -[2023-10-15 18:20:23,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182714368. Throughput: 0: 1812.4, 1: 1798.8. Samples: 45681532. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:23,442][51532] Avg episode reward: [(0, '80.120'), (1, '71.230')] -[2023-10-15 18:20:25,898][52866] Updated weights for policy 1, policy_version 89350 (0.0007) -[2023-10-15 18:20:25,996][52833] Updated weights for policy 0, policy_version 89090 (0.0008) -[2023-10-15 18:20:26,262][52866] Updated weights for policy 1, policy_version 89360 (0.0007) -[2023-10-15 18:20:26,362][52833] Updated weights for policy 0, policy_version 89100 (0.0008) -[2023-10-15 18:20:26,622][52866] Updated weights for policy 1, policy_version 89370 (0.0009) -[2023-10-15 18:20:26,734][52833] Updated weights for policy 0, policy_version 89110 (0.0007) -[2023-10-15 18:20:27,097][52833] Updated weights for policy 0, policy_version 89120 (0.0007) -[2023-10-15 18:20:28,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182779904. Throughput: 0: 1785.7, 1: 1789.2. Samples: 45702438. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:28,442][51532] Avg episode reward: [(0, '80.730'), (1, '69.100')] -[2023-10-15 18:20:30,343][52866] Updated weights for policy 1, policy_version 89380 (0.0008) -[2023-10-15 18:20:30,705][52866] Updated weights for policy 1, policy_version 89390 (0.0007) -[2023-10-15 18:20:31,070][52866] Updated weights for policy 1, policy_version 89400 (0.0008) -[2023-10-15 18:20:31,234][52833] Updated weights for policy 0, policy_version 89130 (0.0011) -[2023-10-15 18:20:31,616][52833] Updated weights for policy 0, policy_version 89140 (0.0011) -[2023-10-15 18:20:31,976][52833] Updated weights for policy 0, policy_version 89150 (0.0010) -[2023-10-15 18:20:33,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182845440. Throughput: 0: 1806.2, 1: 1800.4. Samples: 45714056. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:33,442][51532] Avg episode reward: [(0, '78.220'), (1, '72.600')] -[2023-10-15 18:20:35,004][52866] Updated weights for policy 1, policy_version 89410 (0.0009) -[2023-10-15 18:20:35,371][52866] Updated weights for policy 1, policy_version 89420 (0.0009) -[2023-10-15 18:20:35,729][52833] Updated weights for policy 0, policy_version 89160 (0.0008) -[2023-10-15 18:20:35,739][52866] Updated weights for policy 1, policy_version 89430 (0.0009) -[2023-10-15 18:20:36,092][52833] Updated weights for policy 0, policy_version 89170 (0.0007) -[2023-10-15 18:20:36,102][52866] Updated weights for policy 1, policy_version 89440 (0.0008) -[2023-10-15 18:20:36,467][52833] Updated weights for policy 0, policy_version 89180 (0.0010) -[2023-10-15 18:20:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 182910976. Throughput: 0: 1781.9, 1: 1793.8. Samples: 45734464. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:38,442][51532] Avg episode reward: [(0, '75.990'), (1, '71.770')] -[2023-10-15 18:20:40,113][52866] Updated weights for policy 1, policy_version 89450 (0.0009) -[2023-10-15 18:20:40,270][52833] Updated weights for policy 0, policy_version 89190 (0.0008) -[2023-10-15 18:20:40,478][52866] Updated weights for policy 1, policy_version 89460 (0.0008) -[2023-10-15 18:20:40,639][52833] Updated weights for policy 0, policy_version 89200 (0.0007) -[2023-10-15 18:20:40,845][52866] Updated weights for policy 1, policy_version 89470 (0.0007) -[2023-10-15 18:20:41,003][52833] Updated weights for policy 0, policy_version 89210 (0.0007) -[2023-10-15 18:20:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182976512. Throughput: 0: 1785.8, 1: 1790.2. Samples: 45756748. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:43,442][51532] Avg episode reward: [(0, '77.690'), (1, '74.070')] -[2023-10-15 18:20:44,501][52866] Updated weights for policy 1, policy_version 89480 (0.0008) -[2023-10-15 18:20:44,635][52833] Updated weights for policy 0, policy_version 89220 (0.0007) -[2023-10-15 18:20:44,879][52866] Updated weights for policy 1, policy_version 89490 (0.0010) -[2023-10-15 18:20:44,994][52833] Updated weights for policy 0, policy_version 89230 (0.0007) -[2023-10-15 18:20:45,234][52866] Updated weights for policy 1, policy_version 89500 (0.0007) -[2023-10-15 18:20:45,367][52833] Updated weights for policy 0, policy_version 89240 (0.0007) -[2023-10-15 18:20:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 183042048. Throughput: 0: 1788.2, 1: 1790.8. Samples: 45766732. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:48,442][51532] Avg episode reward: [(0, '74.750'), (1, '73.480')] -[2023-10-15 18:20:48,961][52833] Updated weights for policy 0, policy_version 89250 (0.0007) -[2023-10-15 18:20:48,987][52866] Updated weights for policy 1, policy_version 89510 (0.0007) -[2023-10-15 18:20:49,335][52833] Updated weights for policy 0, policy_version 89260 (0.0008) -[2023-10-15 18:20:49,354][52866] Updated weights for policy 1, policy_version 89520 (0.0007) -[2023-10-15 18:20:49,706][52833] Updated weights for policy 0, policy_version 89270 (0.0009) -[2023-10-15 18:20:49,716][52866] Updated weights for policy 1, policy_version 89530 (0.0008) -[2023-10-15 18:20:50,070][52833] Updated weights for policy 0, policy_version 89280 (0.0009) -[2023-10-15 18:20:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 183107584. Throughput: 0: 1791.8, 1: 1789.3. Samples: 45789160. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:53,441][51532] Avg episode reward: [(0, '75.650'), (1, '73.370')] -[2023-10-15 18:20:53,521][52866] Updated weights for policy 1, policy_version 89540 (0.0009) -[2023-10-15 18:20:53,638][52833] Updated weights for policy 0, policy_version 89290 (0.0008) -[2023-10-15 18:20:53,885][52866] Updated weights for policy 1, policy_version 89550 (0.0008) -[2023-10-15 18:20:54,007][52833] Updated weights for policy 0, policy_version 89300 (0.0007) -[2023-10-15 18:20:54,245][52866] Updated weights for policy 1, policy_version 89560 (0.0008) -[2023-10-15 18:20:54,377][52833] Updated weights for policy 0, policy_version 89310 (0.0007) -[2023-10-15 18:20:58,052][52833] Updated weights for policy 0, policy_version 89320 (0.0007) -[2023-10-15 18:20:58,060][52866] Updated weights for policy 1, policy_version 89570 (0.0009) -[2023-10-15 18:20:58,426][52833] Updated weights for policy 0, policy_version 89330 (0.0007) -[2023-10-15 18:20:58,427][52866] Updated weights for policy 1, policy_version 89580 (0.0008) -[2023-10-15 18:20:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 183173120. Throughput: 0: 1802.7, 1: 1810.9. Samples: 45811668. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:20:58,442][51532] Avg episode reward: [(0, '74.820'), (1, '70.580')] -[2023-10-15 18:20:58,794][52866] Updated weights for policy 1, policy_version 89590 (0.0007) -[2023-10-15 18:20:58,803][52833] Updated weights for policy 0, policy_version 89340 (0.0008) -[2023-10-15 18:20:59,159][52866] Updated weights for policy 1, policy_version 89600 (0.0009) -[2023-10-15 18:21:02,631][52833] Updated weights for policy 0, policy_version 89350 (0.0008) -[2023-10-15 18:21:02,994][52866] Updated weights for policy 1, policy_version 89610 (0.0008) -[2023-10-15 18:21:02,998][52833] Updated weights for policy 0, policy_version 89360 (0.0008) -[2023-10-15 18:21:03,357][52866] Updated weights for policy 1, policy_version 89620 (0.0009) -[2023-10-15 18:21:03,359][52833] Updated weights for policy 0, policy_version 89370 (0.0010) -[2023-10-15 18:21:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 183238656. Throughput: 0: 1788.3, 1: 1784.9. Samples: 45821478. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) -[2023-10-15 18:21:03,441][51532] Avg episode reward: [(0, '76.320'), (1, '69.350')] -[2023-10-15 18:21:03,719][52866] Updated weights for policy 1, policy_version 89630 (0.0008) -[2023-10-15 18:21:07,035][52833] Updated weights for policy 0, policy_version 89380 (0.0007) -[2023-10-15 18:21:07,404][52833] Updated weights for policy 0, policy_version 89390 (0.0007) -[2023-10-15 18:21:07,598][52866] Updated weights for policy 1, policy_version 89640 (0.0008) -[2023-10-15 18:21:07,772][52833] Updated weights for policy 0, policy_version 89400 (0.0008) -[2023-10-15 18:21:07,962][52866] Updated weights for policy 1, policy_version 89650 (0.0007) -[2023-10-15 18:21:08,321][52866] Updated weights for policy 1, policy_version 89660 (0.0008) -[2023-10-15 18:21:08,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 183336960. Throughput: 0: 1802.8, 1: 1800.2. Samples: 45843668. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:08,441][51532] Avg episode reward: [(0, '76.600'), (1, '69.830')] -[2023-10-15 18:21:11,565][52833] Updated weights for policy 0, policy_version 89410 (0.0007) -[2023-10-15 18:21:11,930][52833] Updated weights for policy 0, policy_version 89420 (0.0009) -[2023-10-15 18:21:12,054][52866] Updated weights for policy 1, policy_version 89670 (0.0008) -[2023-10-15 18:21:12,298][52833] Updated weights for policy 0, policy_version 89430 (0.0007) -[2023-10-15 18:21:12,410][52866] Updated weights for policy 1, policy_version 89680 (0.0008) -[2023-10-15 18:21:12,666][52833] Updated weights for policy 0, policy_version 89440 (0.0007) -[2023-10-15 18:21:12,770][52866] Updated weights for policy 1, policy_version 89690 (0.0007) -[2023-10-15 18:21:13,441][51532] Fps is (10 sec: 19660.2, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 183435264. Throughput: 0: 1795.3, 1: 1782.8. Samples: 45863456. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:13,442][51532] Avg episode reward: [(0, '75.930'), (1, '73.360')] -[2023-10-15 18:21:16,283][52866] Updated weights for policy 1, policy_version 89700 (0.0009) -[2023-10-15 18:21:16,514][52833] Updated weights for policy 0, policy_version 89450 (0.0008) -[2023-10-15 18:21:16,649][52866] Updated weights for policy 1, policy_version 89710 (0.0009) -[2023-10-15 18:21:16,885][52833] Updated weights for policy 0, policy_version 89460 (0.0009) -[2023-10-15 18:21:17,014][52866] Updated weights for policy 1, policy_version 89720 (0.0008) -[2023-10-15 18:21:17,252][52833] Updated weights for policy 0, policy_version 89470 (0.0010) -[2023-10-15 18:21:18,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 183500800. Throughput: 0: 1803.6, 1: 1805.2. Samples: 45876454. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:18,442][51532] Avg episode reward: [(0, '75.450'), (1, '69.440')] -[2023-10-15 18:21:20,850][52866] Updated weights for policy 1, policy_version 89730 (0.0007) -[2023-10-15 18:21:21,016][52833] Updated weights for policy 0, policy_version 89480 (0.0007) -[2023-10-15 18:21:21,210][52866] Updated weights for policy 1, policy_version 89740 (0.0007) -[2023-10-15 18:21:21,386][52833] Updated weights for policy 0, policy_version 89490 (0.0007) -[2023-10-15 18:21:21,576][52866] Updated weights for policy 1, policy_version 89750 (0.0008) -[2023-10-15 18:21:21,751][52833] Updated weights for policy 0, policy_version 89500 (0.0008) -[2023-10-15 18:21:21,946][52866] Updated weights for policy 1, policy_version 89760 (0.0010) -[2023-10-15 18:21:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183566336. Throughput: 0: 1808.2, 1: 1784.4. Samples: 45896130. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:23,441][51532] Avg episode reward: [(0, '81.410'), (1, '70.680')] -[2023-10-15 18:21:25,652][52833] Updated weights for policy 0, policy_version 89510 (0.0008) -[2023-10-15 18:21:25,851][52866] Updated weights for policy 1, policy_version 89770 (0.0009) -[2023-10-15 18:21:26,018][52833] Updated weights for policy 0, policy_version 89520 (0.0007) -[2023-10-15 18:21:26,216][52866] Updated weights for policy 1, policy_version 89780 (0.0007) -[2023-10-15 18:21:26,393][52833] Updated weights for policy 0, policy_version 89530 (0.0008) -[2023-10-15 18:21:26,584][52866] Updated weights for policy 1, policy_version 89790 (0.0009) -[2023-10-15 18:21:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183631872. Throughput: 0: 1798.3, 1: 1784.1. Samples: 45917956. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:28,441][51532] Avg episode reward: [(0, '81.840'), (1, '69.230')] -[2023-10-15 18:21:30,021][52833] Updated weights for policy 0, policy_version 89540 (0.0009) -[2023-10-15 18:21:30,328][52866] Updated weights for policy 1, policy_version 89800 (0.0007) -[2023-10-15 18:21:30,403][52833] Updated weights for policy 0, policy_version 89550 (0.0009) -[2023-10-15 18:21:30,687][52866] Updated weights for policy 1, policy_version 89810 (0.0010) -[2023-10-15 18:21:30,776][52833] Updated weights for policy 0, policy_version 89560 (0.0007) -[2023-10-15 18:21:31,047][52866] Updated weights for policy 1, policy_version 89820 (0.0008) -[2023-10-15 18:21:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183697408. Throughput: 0: 1801.7, 1: 1788.7. Samples: 45928302. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:33,441][51532] Avg episode reward: [(0, '85.110'), (1, '69.900')] -[2023-10-15 18:21:34,662][52833] Updated weights for policy 0, policy_version 89570 (0.0009) -[2023-10-15 18:21:34,950][52866] Updated weights for policy 1, policy_version 89830 (0.0010) -[2023-10-15 18:21:35,033][52833] Updated weights for policy 0, policy_version 89580 (0.0008) -[2023-10-15 18:21:35,316][52866] Updated weights for policy 1, policy_version 89840 (0.0009) -[2023-10-15 18:21:35,395][52833] Updated weights for policy 0, policy_version 89590 (0.0007) -[2023-10-15 18:21:35,677][52866] Updated weights for policy 1, policy_version 89850 (0.0008) -[2023-10-15 18:21:35,765][52833] Updated weights for policy 0, policy_version 89600 (0.0009) -[2023-10-15 18:21:38,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183762944. Throughput: 0: 1793.9, 1: 1775.3. Samples: 45949778. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:38,442][51532] Avg episode reward: [(0, '86.000'), (1, '71.190')] -[2023-10-15 18:21:39,322][52833] Updated weights for policy 0, policy_version 89610 (0.0010) -[2023-10-15 18:21:39,521][52866] Updated weights for policy 1, policy_version 89860 (0.0008) -[2023-10-15 18:21:39,685][52833] Updated weights for policy 0, policy_version 89620 (0.0007) -[2023-10-15 18:21:39,886][52866] Updated weights for policy 1, policy_version 89870 (0.0011) -[2023-10-15 18:21:40,061][52833] Updated weights for policy 0, policy_version 89630 (0.0009) -[2023-10-15 18:21:40,252][52866] Updated weights for policy 1, policy_version 89880 (0.0010) -[2023-10-15 18:21:43,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 183828480. Throughput: 0: 1793.9, 1: 1768.9. Samples: 45971992. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:43,442][51532] Avg episode reward: [(0, '86.170'), (1, '73.970')] -[2023-10-15 18:21:43,451][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000089632_91783168.pth... -[2023-10-15 18:21:43,451][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000089888_92045312.pth... -[2023-10-15 18:21:43,490][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000087968_90079232.pth -[2023-10-15 18:21:43,492][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000088224_90341376.pth -[2023-10-15 18:21:43,956][52833] Updated weights for policy 0, policy_version 89640 (0.0007) -[2023-10-15 18:21:43,998][52866] Updated weights for policy 1, policy_version 89890 (0.0010) -[2023-10-15 18:21:44,323][52833] Updated weights for policy 0, policy_version 89650 (0.0008) -[2023-10-15 18:21:44,365][52866] Updated weights for policy 1, policy_version 89900 (0.0008) -[2023-10-15 18:21:44,687][52833] Updated weights for policy 0, policy_version 89660 (0.0009) -[2023-10-15 18:21:44,729][52866] Updated weights for policy 1, policy_version 89910 (0.0007) -[2023-10-15 18:21:45,089][52866] Updated weights for policy 1, policy_version 89920 (0.0009) -[2023-10-15 18:21:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 183894016. Throughput: 0: 1789.9, 1: 1767.7. Samples: 45981570. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:48,442][51532] Avg episode reward: [(0, '86.760'), (1, '74.740')] -[2023-10-15 18:21:48,452][52833] Updated weights for policy 0, policy_version 89670 (0.0009) -[2023-10-15 18:21:48,811][52833] Updated weights for policy 0, policy_version 89680 (0.0009) -[2023-10-15 18:21:48,905][52866] Updated weights for policy 1, policy_version 89930 (0.0008) -[2023-10-15 18:21:49,183][52833] Updated weights for policy 0, policy_version 89690 (0.0007) -[2023-10-15 18:21:49,272][52866] Updated weights for policy 1, policy_version 89940 (0.0008) -[2023-10-15 18:21:49,399][52410] Saving new best policy, reward=86.760! -[2023-10-15 18:21:49,632][52866] Updated weights for policy 1, policy_version 89950 (0.0007) -[2023-10-15 18:21:53,060][52833] Updated weights for policy 0, policy_version 89700 (0.0008) -[2023-10-15 18:21:53,220][52866] Updated weights for policy 1, policy_version 89960 (0.0008) -[2023-10-15 18:21:53,425][52833] Updated weights for policy 0, policy_version 89710 (0.0007) -[2023-10-15 18:21:53,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 183959552. Throughput: 0: 1781.5, 1: 1782.3. Samples: 46004038. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:53,441][51532] Avg episode reward: [(0, '86.560'), (1, '75.660')] -[2023-10-15 18:21:53,582][52866] Updated weights for policy 1, policy_version 89970 (0.0007) -[2023-10-15 18:21:53,790][52833] Updated weights for policy 0, policy_version 89720 (0.0007) -[2023-10-15 18:21:53,947][52866] Updated weights for policy 1, policy_version 89980 (0.0007) -[2023-10-15 18:21:57,550][52833] Updated weights for policy 0, policy_version 89730 (0.0008) -[2023-10-15 18:21:57,708][52866] Updated weights for policy 1, policy_version 89990 (0.0007) -[2023-10-15 18:21:57,919][52833] Updated weights for policy 0, policy_version 89740 (0.0007) -[2023-10-15 18:21:58,067][52866] Updated weights for policy 1, policy_version 90000 (0.0007) -[2023-10-15 18:21:58,284][52833] Updated weights for policy 0, policy_version 89750 (0.0008) -[2023-10-15 18:21:58,437][52866] Updated weights for policy 1, policy_version 90010 (0.0009) -[2023-10-15 18:21:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 184025088. Throughput: 0: 1799.1, 1: 1799.9. Samples: 46025408. Policy #0 lag: (min: 29.0, avg: 29.9, max: 48.0) -[2023-10-15 18:21:58,441][51532] Avg episode reward: [(0, '86.660'), (1, '75.080')] -[2023-10-15 18:21:58,653][52833] Updated weights for policy 0, policy_version 89760 (0.0008) -[2023-10-15 18:22:02,201][52866] Updated weights for policy 1, policy_version 90020 (0.0008) -[2023-10-15 18:22:02,381][52833] Updated weights for policy 0, policy_version 89770 (0.0007) -[2023-10-15 18:22:02,560][52866] Updated weights for policy 1, policy_version 90030 (0.0008) -[2023-10-15 18:22:02,749][52833] Updated weights for policy 0, policy_version 89780 (0.0008) -[2023-10-15 18:22:02,913][52866] Updated weights for policy 1, policy_version 90040 (0.0008) -[2023-10-15 18:22:03,112][52833] Updated weights for policy 0, policy_version 89790 (0.0009) -[2023-10-15 18:22:03,441][51532] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 184156160. Throughput: 0: 1779.2, 1: 1777.0. Samples: 46036482. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:03,442][51532] Avg episode reward: [(0, '85.020'), (1, '75.260')] -[2023-10-15 18:22:06,619][52866] Updated weights for policy 1, policy_version 90050 (0.0007) -[2023-10-15 18:22:06,809][52833] Updated weights for policy 0, policy_version 89800 (0.0009) -[2023-10-15 18:22:06,978][52866] Updated weights for policy 1, policy_version 90060 (0.0007) -[2023-10-15 18:22:07,173][52833] Updated weights for policy 0, policy_version 89810 (0.0007) -[2023-10-15 18:22:07,340][52866] Updated weights for policy 1, policy_version 90070 (0.0007) -[2023-10-15 18:22:07,538][52833] Updated weights for policy 0, policy_version 89820 (0.0008) -[2023-10-15 18:22:07,706][52866] Updated weights for policy 1, policy_version 90080 (0.0007) -[2023-10-15 18:22:08,441][51532] Fps is (10 sec: 19660.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 184221696. Throughput: 0: 1799.9, 1: 1796.2. Samples: 46057956. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:08,442][51532] Avg episode reward: [(0, '84.400'), (1, '75.360')] -[2023-10-15 18:22:11,197][52833] Updated weights for policy 0, policy_version 89830 (0.0009) -[2023-10-15 18:22:11,538][52866] Updated weights for policy 1, policy_version 90090 (0.0007) -[2023-10-15 18:22:11,571][52833] Updated weights for policy 0, policy_version 89840 (0.0009) -[2023-10-15 18:22:11,905][52866] Updated weights for policy 1, policy_version 90100 (0.0008) -[2023-10-15 18:22:11,933][52833] Updated weights for policy 0, policy_version 89850 (0.0008) -[2023-10-15 18:22:12,269][52866] Updated weights for policy 1, policy_version 90110 (0.0007) -[2023-10-15 18:22:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184287232. Throughput: 0: 1784.5, 1: 1786.0. Samples: 46078628. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:13,442][51532] Avg episode reward: [(0, '84.590'), (1, '75.760')] -[2023-10-15 18:22:15,786][52833] Updated weights for policy 0, policy_version 89860 (0.0008) -[2023-10-15 18:22:16,092][52866] Updated weights for policy 1, policy_version 90120 (0.0007) -[2023-10-15 18:22:16,145][52833] Updated weights for policy 0, policy_version 89870 (0.0007) -[2023-10-15 18:22:16,450][52866] Updated weights for policy 1, policy_version 90130 (0.0007) -[2023-10-15 18:22:16,507][52833] Updated weights for policy 0, policy_version 89880 (0.0008) -[2023-10-15 18:22:16,821][52866] Updated weights for policy 1, policy_version 90140 (0.0007) -[2023-10-15 18:22:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184352768. Throughput: 0: 1806.5, 1: 1805.1. Samples: 46090824. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:18,442][51532] Avg episode reward: [(0, '84.910'), (1, '75.500')] -[2023-10-15 18:22:20,132][52833] Updated weights for policy 0, policy_version 89890 (0.0007) -[2023-10-15 18:22:20,475][52866] Updated weights for policy 1, policy_version 90150 (0.0008) -[2023-10-15 18:22:20,494][52833] Updated weights for policy 0, policy_version 89900 (0.0009) -[2023-10-15 18:22:20,832][52866] Updated weights for policy 1, policy_version 90160 (0.0007) -[2023-10-15 18:22:20,858][52833] Updated weights for policy 0, policy_version 89910 (0.0008) -[2023-10-15 18:22:21,199][52866] Updated weights for policy 1, policy_version 90170 (0.0008) -[2023-10-15 18:22:21,229][52833] Updated weights for policy 0, policy_version 89920 (0.0010) -[2023-10-15 18:22:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 184418304. Throughput: 0: 1790.6, 1: 1795.6. Samples: 46111158. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:23,442][51532] Avg episode reward: [(0, '84.310'), (1, '73.260')] -[2023-10-15 18:22:25,064][52833] Updated weights for policy 0, policy_version 89930 (0.0010) -[2023-10-15 18:22:25,122][52866] Updated weights for policy 1, policy_version 90180 (0.0008) -[2023-10-15 18:22:25,425][52833] Updated weights for policy 0, policy_version 89940 (0.0007) -[2023-10-15 18:22:25,484][52866] Updated weights for policy 1, policy_version 90190 (0.0008) -[2023-10-15 18:22:25,786][52833] Updated weights for policy 0, policy_version 89950 (0.0007) -[2023-10-15 18:22:25,857][52866] Updated weights for policy 1, policy_version 90200 (0.0007) -[2023-10-15 18:22:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 184483840. Throughput: 0: 1789.1, 1: 1794.5. Samples: 46133252. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:28,442][51532] Avg episode reward: [(0, '80.770'), (1, '73.940')] -[2023-10-15 18:22:29,584][52833] Updated weights for policy 0, policy_version 89960 (0.0009) -[2023-10-15 18:22:29,606][52866] Updated weights for policy 1, policy_version 90210 (0.0009) -[2023-10-15 18:22:29,952][52833] Updated weights for policy 0, policy_version 89970 (0.0008) -[2023-10-15 18:22:29,972][52866] Updated weights for policy 1, policy_version 90220 (0.0008) -[2023-10-15 18:22:30,310][52833] Updated weights for policy 0, policy_version 89980 (0.0008) -[2023-10-15 18:22:30,340][52866] Updated weights for policy 1, policy_version 90230 (0.0008) -[2023-10-15 18:22:30,709][52866] Updated weights for policy 1, policy_version 90240 (0.0008) -[2023-10-15 18:22:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 184549376. Throughput: 0: 1790.3, 1: 1797.2. Samples: 46143010. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:33,442][51532] Avg episode reward: [(0, '76.540'), (1, '73.030')] -[2023-10-15 18:22:34,043][52833] Updated weights for policy 0, policy_version 89990 (0.0008) -[2023-10-15 18:22:34,409][52833] Updated weights for policy 0, policy_version 90000 (0.0008) -[2023-10-15 18:22:34,455][52866] Updated weights for policy 1, policy_version 90250 (0.0009) -[2023-10-15 18:22:34,781][52833] Updated weights for policy 0, policy_version 90010 (0.0008) -[2023-10-15 18:22:34,813][52866] Updated weights for policy 1, policy_version 90260 (0.0009) -[2023-10-15 18:22:35,178][52866] Updated weights for policy 1, policy_version 90270 (0.0009) -[2023-10-15 18:22:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 184614912. Throughput: 0: 1798.1, 1: 1789.5. Samples: 46165480. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:38,442][51532] Avg episode reward: [(0, '80.380'), (1, '71.890')] -[2023-10-15 18:22:38,468][52833] Updated weights for policy 0, policy_version 90020 (0.0007) -[2023-10-15 18:22:38,834][52833] Updated weights for policy 0, policy_version 90030 (0.0009) -[2023-10-15 18:22:39,116][52866] Updated weights for policy 1, policy_version 90280 (0.0008) -[2023-10-15 18:22:39,200][52833] Updated weights for policy 0, policy_version 90040 (0.0008) -[2023-10-15 18:22:39,479][52866] Updated weights for policy 1, policy_version 90290 (0.0007) -[2023-10-15 18:22:39,850][52866] Updated weights for policy 1, policy_version 90300 (0.0010) -[2023-10-15 18:22:43,074][52833] Updated weights for policy 0, policy_version 90050 (0.0009) -[2023-10-15 18:22:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 184680448. Throughput: 0: 1808.2, 1: 1802.6. Samples: 46187894. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:43,441][51532] Avg episode reward: [(0, '79.740'), (1, '67.570')] -[2023-10-15 18:22:43,447][52833] Updated weights for policy 0, policy_version 90060 (0.0008) -[2023-10-15 18:22:43,565][52866] Updated weights for policy 1, policy_version 90310 (0.0009) -[2023-10-15 18:22:43,821][52833] Updated weights for policy 0, policy_version 90070 (0.0008) -[2023-10-15 18:22:43,938][52866] Updated weights for policy 1, policy_version 90320 (0.0007) -[2023-10-15 18:22:44,180][52833] Updated weights for policy 0, policy_version 90080 (0.0008) -[2023-10-15 18:22:44,296][52866] Updated weights for policy 1, policy_version 90330 (0.0008) -[2023-10-15 18:22:48,078][52833] Updated weights for policy 0, policy_version 90090 (0.0008) -[2023-10-15 18:22:48,093][52866] Updated weights for policy 1, policy_version 90340 (0.0009) -[2023-10-15 18:22:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 184745984. Throughput: 0: 1792.3, 1: 1788.2. Samples: 46197602. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:48,441][51532] Avg episode reward: [(0, '78.760'), (1, '64.050')] -[2023-10-15 18:22:48,447][52833] Updated weights for policy 0, policy_version 90100 (0.0009) -[2023-10-15 18:22:48,450][52866] Updated weights for policy 1, policy_version 90350 (0.0009) -[2023-10-15 18:22:48,813][52866] Updated weights for policy 1, policy_version 90360 (0.0008) -[2023-10-15 18:22:48,818][52833] Updated weights for policy 0, policy_version 90110 (0.0008) -[2023-10-15 18:22:52,497][52833] Updated weights for policy 0, policy_version 90120 (0.0008) -[2023-10-15 18:22:52,512][52866] Updated weights for policy 1, policy_version 90370 (0.0007) -[2023-10-15 18:22:52,854][52833] Updated weights for policy 0, policy_version 90130 (0.0008) -[2023-10-15 18:22:52,884][52866] Updated weights for policy 1, policy_version 90380 (0.0007) -[2023-10-15 18:22:53,229][52833] Updated weights for policy 0, policy_version 90140 (0.0008) -[2023-10-15 18:22:53,250][52866] Updated weights for policy 1, policy_version 90390 (0.0007) -[2023-10-15 18:22:53,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 184844288. Throughput: 0: 1795.8, 1: 1804.8. Samples: 46219982. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:53,442][51532] Avg episode reward: [(0, '78.870'), (1, '64.870')] -[2023-10-15 18:22:53,615][52866] Updated weights for policy 1, policy_version 90400 (0.0007) -[2023-10-15 18:22:56,974][52833] Updated weights for policy 0, policy_version 90150 (0.0009) -[2023-10-15 18:22:57,346][52833] Updated weights for policy 0, policy_version 90160 (0.0007) -[2023-10-15 18:22:57,458][52866] Updated weights for policy 1, policy_version 90410 (0.0007) -[2023-10-15 18:22:57,713][52833] Updated weights for policy 0, policy_version 90170 (0.0011) -[2023-10-15 18:22:57,826][52866] Updated weights for policy 1, policy_version 90420 (0.0007) -[2023-10-15 18:22:58,193][52866] Updated weights for policy 1, policy_version 90430 (0.0007) -[2023-10-15 18:22:58,441][51532] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 184942592. Throughput: 0: 1790.7, 1: 1796.4. Samples: 46240046. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-15 18:22:58,442][51532] Avg episode reward: [(0, '79.660'), (1, '63.450')] -[2023-10-15 18:23:01,375][52833] Updated weights for policy 0, policy_version 90180 (0.0009) -[2023-10-15 18:23:01,742][52833] Updated weights for policy 0, policy_version 90190 (0.0008) -[2023-10-15 18:23:02,011][52866] Updated weights for policy 1, policy_version 90440 (0.0008) -[2023-10-15 18:23:02,107][52833] Updated weights for policy 0, policy_version 90200 (0.0007) -[2023-10-15 18:23:02,379][52866] Updated weights for policy 1, policy_version 90450 (0.0009) -[2023-10-15 18:23:02,748][52866] Updated weights for policy 1, policy_version 90460 (0.0008) -[2023-10-15 18:23:03,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185008128. Throughput: 0: 1796.6, 1: 1792.6. Samples: 46252340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:03,441][51532] Avg episode reward: [(0, '77.840'), (1, '61.910')] -[2023-10-15 18:23:05,787][52833] Updated weights for policy 0, policy_version 90210 (0.0007) -[2023-10-15 18:23:06,153][52833] Updated weights for policy 0, policy_version 90220 (0.0008) -[2023-10-15 18:23:06,411][52866] Updated weights for policy 1, policy_version 90470 (0.0008) -[2023-10-15 18:23:06,527][52833] Updated weights for policy 0, policy_version 90230 (0.0007) -[2023-10-15 18:23:06,768][52866] Updated weights for policy 1, policy_version 90480 (0.0009) -[2023-10-15 18:23:06,894][52833] Updated weights for policy 0, policy_version 90240 (0.0009) -[2023-10-15 18:23:07,146][52866] Updated weights for policy 1, policy_version 90490 (0.0008) -[2023-10-15 18:23:08,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185073664. Throughput: 0: 1791.8, 1: 1792.1. Samples: 46272434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:08,442][51532] Avg episode reward: [(0, '77.750'), (1, '61.440')] -[2023-10-15 18:23:10,611][52833] Updated weights for policy 0, policy_version 90250 (0.0010) -[2023-10-15 18:23:10,870][52866] Updated weights for policy 1, policy_version 90500 (0.0010) -[2023-10-15 18:23:10,980][52833] Updated weights for policy 0, policy_version 90260 (0.0009) -[2023-10-15 18:23:11,242][52866] Updated weights for policy 1, policy_version 90510 (0.0007) -[2023-10-15 18:23:11,342][52833] Updated weights for policy 0, policy_version 90270 (0.0008) -[2023-10-15 18:23:11,597][52866] Updated weights for policy 1, policy_version 90520 (0.0008) -[2023-10-15 18:23:13,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185139200. Throughput: 0: 1790.7, 1: 1789.0. Samples: 46294340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:13,442][51532] Avg episode reward: [(0, '80.930'), (1, '60.030')] -[2023-10-15 18:23:15,176][52833] Updated weights for policy 0, policy_version 90280 (0.0010) -[2023-10-15 18:23:15,217][52866] Updated weights for policy 1, policy_version 90530 (0.0009) -[2023-10-15 18:23:15,535][52833] Updated weights for policy 0, policy_version 90290 (0.0008) -[2023-10-15 18:23:15,582][52866] Updated weights for policy 1, policy_version 90540 (0.0008) -[2023-10-15 18:23:15,902][52833] Updated weights for policy 0, policy_version 90300 (0.0007) -[2023-10-15 18:23:15,958][52866] Updated weights for policy 1, policy_version 90550 (0.0010) -[2023-10-15 18:23:16,320][52866] Updated weights for policy 1, policy_version 90560 (0.0010) -[2023-10-15 18:23:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185204736. Throughput: 0: 1794.9, 1: 1799.9. Samples: 46304776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:18,442][51532] Avg episode reward: [(0, '79.250'), (1, '58.680')] -[2023-10-15 18:23:19,746][52833] Updated weights for policy 0, policy_version 90310 (0.0008) -[2023-10-15 18:23:20,115][52833] Updated weights for policy 0, policy_version 90320 (0.0009) -[2023-10-15 18:23:20,248][52866] Updated weights for policy 1, policy_version 90570 (0.0007) -[2023-10-15 18:23:20,490][52833] Updated weights for policy 0, policy_version 90330 (0.0008) -[2023-10-15 18:23:20,620][52866] Updated weights for policy 1, policy_version 90580 (0.0007) -[2023-10-15 18:23:20,986][52866] Updated weights for policy 1, policy_version 90590 (0.0007) -[2023-10-15 18:23:23,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 185270272. Throughput: 0: 1784.3, 1: 1785.9. Samples: 46326138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:23,441][51532] Avg episode reward: [(0, '74.480'), (1, '57.570')] -[2023-10-15 18:23:24,064][52833] Updated weights for policy 0, policy_version 90340 (0.0007) -[2023-10-15 18:23:24,427][52833] Updated weights for policy 0, policy_version 90350 (0.0007) -[2023-10-15 18:23:24,667][52866] Updated weights for policy 1, policy_version 90600 (0.0007) -[2023-10-15 18:23:24,795][52833] Updated weights for policy 0, policy_version 90360 (0.0009) -[2023-10-15 18:23:25,033][52866] Updated weights for policy 1, policy_version 90610 (0.0009) -[2023-10-15 18:23:25,405][52866] Updated weights for policy 1, policy_version 90620 (0.0008) -[2023-10-15 18:23:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 185335808. Throughput: 0: 1786.8, 1: 1786.4. Samples: 46348690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:28,441][51532] Avg episode reward: [(0, '74.410'), (1, '58.960')] -[2023-10-15 18:23:28,554][52833] Updated weights for policy 0, policy_version 90370 (0.0008) -[2023-10-15 18:23:28,917][52833] Updated weights for policy 0, policy_version 90380 (0.0008) -[2023-10-15 18:23:29,281][52866] Updated weights for policy 1, policy_version 90630 (0.0009) -[2023-10-15 18:23:29,284][52833] Updated weights for policy 0, policy_version 90390 (0.0007) -[2023-10-15 18:23:29,657][52833] Updated weights for policy 0, policy_version 90400 (0.0008) -[2023-10-15 18:23:29,659][52866] Updated weights for policy 1, policy_version 90640 (0.0009) -[2023-10-15 18:23:30,020][52866] Updated weights for policy 1, policy_version 90650 (0.0008) -[2023-10-15 18:23:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 185401344. Throughput: 0: 1788.1, 1: 1786.9. Samples: 46358478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:33,442][51532] Avg episode reward: [(0, '71.070'), (1, '56.690')] -[2023-10-15 18:23:33,447][52833] Updated weights for policy 0, policy_version 90410 (0.0009) -[2023-10-15 18:23:33,755][52866] Updated weights for policy 1, policy_version 90660 (0.0009) -[2023-10-15 18:23:33,815][52833] Updated weights for policy 0, policy_version 90420 (0.0008) -[2023-10-15 18:23:34,118][52866] Updated weights for policy 1, policy_version 90670 (0.0008) -[2023-10-15 18:23:34,184][52833] Updated weights for policy 0, policy_version 90430 (0.0008) -[2023-10-15 18:23:34,486][52866] Updated weights for policy 1, policy_version 90680 (0.0009) -[2023-10-15 18:23:37,976][52833] Updated weights for policy 0, policy_version 90440 (0.0007) -[2023-10-15 18:23:38,232][52866] Updated weights for policy 1, policy_version 90690 (0.0009) -[2023-10-15 18:23:38,340][52833] Updated weights for policy 0, policy_version 90450 (0.0007) -[2023-10-15 18:23:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 185466880. Throughput: 0: 1789.3, 1: 1782.1. Samples: 46380696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:38,441][51532] Avg episode reward: [(0, '70.410'), (1, '57.200')] -[2023-10-15 18:23:38,603][52866] Updated weights for policy 1, policy_version 90700 (0.0008) -[2023-10-15 18:23:38,712][52833] Updated weights for policy 0, policy_version 90460 (0.0009) -[2023-10-15 18:23:38,968][52866] Updated weights for policy 1, policy_version 90710 (0.0007) -[2023-10-15 18:23:39,329][52866] Updated weights for policy 1, policy_version 90720 (0.0007) -[2023-10-15 18:23:42,575][52833] Updated weights for policy 0, policy_version 90470 (0.0008) -[2023-10-15 18:23:42,943][52833] Updated weights for policy 0, policy_version 90480 (0.0008) -[2023-10-15 18:23:43,092][52866] Updated weights for policy 1, policy_version 90730 (0.0008) -[2023-10-15 18:23:43,315][52833] Updated weights for policy 0, policy_version 90490 (0.0008) -[2023-10-15 18:23:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 185532416. Throughput: 0: 1802.1, 1: 1800.7. Samples: 46402168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:43,442][51532] Avg episode reward: [(0, '69.040'), (1, '54.530')] -[2023-10-15 18:23:43,462][52866] Updated weights for policy 1, policy_version 90740 (0.0008) -[2023-10-15 18:23:43,526][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000090496_92667904.pth... -[2023-10-15 18:23:43,559][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000088800_90931200.pth -[2023-10-15 18:23:43,827][52866] Updated weights for policy 1, policy_version 90750 (0.0008) -[2023-10-15 18:23:43,896][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000090752_92930048.pth... -[2023-10-15 18:23:43,933][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000089056_91193344.pth -[2023-10-15 18:23:47,134][52833] Updated weights for policy 0, policy_version 90500 (0.0008) -[2023-10-15 18:23:47,465][52866] Updated weights for policy 1, policy_version 90760 (0.0008) -[2023-10-15 18:23:47,499][52833] Updated weights for policy 0, policy_version 90510 (0.0007) -[2023-10-15 18:23:47,822][52866] Updated weights for policy 1, policy_version 90770 (0.0008) -[2023-10-15 18:23:47,874][52833] Updated weights for policy 0, policy_version 90520 (0.0009) -[2023-10-15 18:23:48,184][52866] Updated weights for policy 1, policy_version 90780 (0.0008) -[2023-10-15 18:23:48,441][51532] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14440.2). Total num frames: 185663488. Throughput: 0: 1780.2, 1: 1786.4. Samples: 46412836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:48,441][51532] Avg episode reward: [(0, '68.490'), (1, '56.680')] -[2023-10-15 18:23:51,756][52833] Updated weights for policy 0, policy_version 90530 (0.0008) -[2023-10-15 18:23:52,091][52866] Updated weights for policy 1, policy_version 90790 (0.0008) -[2023-10-15 18:23:52,128][52833] Updated weights for policy 0, policy_version 90540 (0.0008) -[2023-10-15 18:23:52,457][52866] Updated weights for policy 1, policy_version 90800 (0.0009) -[2023-10-15 18:23:52,485][52833] Updated weights for policy 0, policy_version 90550 (0.0007) -[2023-10-15 18:23:52,817][52866] Updated weights for policy 1, policy_version 90810 (0.0007) -[2023-10-15 18:23:52,853][52833] Updated weights for policy 0, policy_version 90560 (0.0007) -[2023-10-15 18:23:53,441][51532] Fps is (10 sec: 19660.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 185729024. Throughput: 0: 1801.1, 1: 1804.4. Samples: 46434682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:53,442][51532] Avg episode reward: [(0, '69.740'), (1, '58.290')] -[2023-10-15 18:23:56,593][52866] Updated weights for policy 1, policy_version 90820 (0.0007) -[2023-10-15 18:23:56,603][52833] Updated weights for policy 0, policy_version 90570 (0.0007) -[2023-10-15 18:23:56,962][52866] Updated weights for policy 1, policy_version 90830 (0.0007) -[2023-10-15 18:23:56,962][52833] Updated weights for policy 0, policy_version 90580 (0.0008) -[2023-10-15 18:23:57,333][52866] Updated weights for policy 1, policy_version 90840 (0.0007) -[2023-10-15 18:23:57,335][52833] Updated weights for policy 0, policy_version 90590 (0.0010) -[2023-10-15 18:23:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185794560. Throughput: 0: 1778.5, 1: 1785.7. Samples: 46454728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:23:58,441][51532] Avg episode reward: [(0, '68.360'), (1, '57.460')] -[2023-10-15 18:24:00,999][52833] Updated weights for policy 0, policy_version 90600 (0.0008) -[2023-10-15 18:24:01,058][52866] Updated weights for policy 1, policy_version 90850 (0.0009) -[2023-10-15 18:24:01,373][52833] Updated weights for policy 0, policy_version 90610 (0.0010) -[2023-10-15 18:24:01,431][52866] Updated weights for policy 1, policy_version 90860 (0.0008) -[2023-10-15 18:24:01,736][52833] Updated weights for policy 0, policy_version 90620 (0.0009) -[2023-10-15 18:24:01,785][52866] Updated weights for policy 1, policy_version 90870 (0.0008) -[2023-10-15 18:24:02,146][52866] Updated weights for policy 1, policy_version 90880 (0.0007) -[2023-10-15 18:24:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185860096. Throughput: 0: 1808.0, 1: 1803.8. Samples: 46467306. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:03,441][51532] Avg episode reward: [(0, '73.270'), (1, '59.760')] -[2023-10-15 18:24:05,616][52833] Updated weights for policy 0, policy_version 90630 (0.0010) -[2023-10-15 18:24:05,943][52866] Updated weights for policy 1, policy_version 90890 (0.0008) -[2023-10-15 18:24:05,985][52833] Updated weights for policy 0, policy_version 90640 (0.0010) -[2023-10-15 18:24:06,298][52866] Updated weights for policy 1, policy_version 90900 (0.0009) -[2023-10-15 18:24:06,352][52833] Updated weights for policy 0, policy_version 90650 (0.0007) -[2023-10-15 18:24:06,665][52866] Updated weights for policy 1, policy_version 90910 (0.0008) -[2023-10-15 18:24:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 185925632. Throughput: 0: 1783.1, 1: 1786.7. Samples: 46486778. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:08,442][51532] Avg episode reward: [(0, '69.970'), (1, '58.150')] -[2023-10-15 18:24:09,952][52833] Updated weights for policy 0, policy_version 90660 (0.0008) -[2023-10-15 18:24:10,313][52833] Updated weights for policy 0, policy_version 90670 (0.0008) -[2023-10-15 18:24:10,329][52866] Updated weights for policy 1, policy_version 90920 (0.0009) -[2023-10-15 18:24:10,686][52833] Updated weights for policy 0, policy_version 90680 (0.0007) -[2023-10-15 18:24:10,698][52866] Updated weights for policy 1, policy_version 90930 (0.0007) -[2023-10-15 18:24:11,055][52866] Updated weights for policy 1, policy_version 90940 (0.0008) -[2023-10-15 18:24:13,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185991168. Throughput: 0: 1786.0, 1: 1789.7. Samples: 46509594. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:13,441][51532] Avg episode reward: [(0, '69.960'), (1, '58.310')] -[2023-10-15 18:24:14,375][52833] Updated weights for policy 0, policy_version 90690 (0.0008) -[2023-10-15 18:24:14,740][52833] Updated weights for policy 0, policy_version 90700 (0.0008) -[2023-10-15 18:24:14,828][52866] Updated weights for policy 1, policy_version 90950 (0.0009) -[2023-10-15 18:24:15,124][52833] Updated weights for policy 0, policy_version 90710 (0.0008) -[2023-10-15 18:24:15,203][52866] Updated weights for policy 1, policy_version 90960 (0.0008) -[2023-10-15 18:24:15,492][52833] Updated weights for policy 0, policy_version 90720 (0.0009) -[2023-10-15 18:24:15,568][52866] Updated weights for policy 1, policy_version 90970 (0.0008) -[2023-10-15 18:24:18,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 186056704. Throughput: 0: 1784.9, 1: 1791.0. Samples: 46519394. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:18,442][51532] Avg episode reward: [(0, '69.490'), (1, '57.600')] -[2023-10-15 18:24:19,139][52833] Updated weights for policy 0, policy_version 90730 (0.0007) -[2023-10-15 18:24:19,352][52866] Updated weights for policy 1, policy_version 90980 (0.0008) -[2023-10-15 18:24:19,506][52833] Updated weights for policy 0, policy_version 90740 (0.0008) -[2023-10-15 18:24:19,715][52866] Updated weights for policy 1, policy_version 90990 (0.0008) -[2023-10-15 18:24:19,875][52833] Updated weights for policy 0, policy_version 90750 (0.0008) -[2023-10-15 18:24:20,074][52866] Updated weights for policy 1, policy_version 91000 (0.0008) -[2023-10-15 18:24:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 186122240. Throughput: 0: 1792.9, 1: 1789.2. Samples: 46541894. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:23,442][51532] Avg episode reward: [(0, '72.710'), (1, '59.940')] -[2023-10-15 18:24:23,637][52833] Updated weights for policy 0, policy_version 90760 (0.0010) -[2023-10-15 18:24:23,765][52866] Updated weights for policy 1, policy_version 91010 (0.0008) -[2023-10-15 18:24:24,014][52833] Updated weights for policy 0, policy_version 90770 (0.0008) -[2023-10-15 18:24:24,136][52866] Updated weights for policy 1, policy_version 91020 (0.0009) -[2023-10-15 18:24:24,382][52833] Updated weights for policy 0, policy_version 90780 (0.0008) -[2023-10-15 18:24:24,500][52866] Updated weights for policy 1, policy_version 91030 (0.0009) -[2023-10-15 18:24:24,869][52866] Updated weights for policy 1, policy_version 91040 (0.0011) -[2023-10-15 18:24:28,100][52833] Updated weights for policy 0, policy_version 90790 (0.0009) -[2023-10-15 18:24:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 186187776. Throughput: 0: 1807.4, 1: 1798.1. Samples: 46564416. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:28,442][51532] Avg episode reward: [(0, '75.460'), (1, '60.310')] -[2023-10-15 18:24:28,471][52833] Updated weights for policy 0, policy_version 90800 (0.0008) -[2023-10-15 18:24:28,760][52866] Updated weights for policy 1, policy_version 91050 (0.0008) -[2023-10-15 18:24:28,836][52833] Updated weights for policy 0, policy_version 90810 (0.0007) -[2023-10-15 18:24:29,134][52866] Updated weights for policy 1, policy_version 91060 (0.0010) -[2023-10-15 18:24:29,501][52866] Updated weights for policy 1, policy_version 91070 (0.0009) -[2023-10-15 18:24:32,719][52833] Updated weights for policy 0, policy_version 90820 (0.0009) -[2023-10-15 18:24:33,095][52833] Updated weights for policy 0, policy_version 90830 (0.0009) -[2023-10-15 18:24:33,259][52866] Updated weights for policy 1, policy_version 91080 (0.0008) -[2023-10-15 18:24:33,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 186253312. Throughput: 0: 1794.4, 1: 1786.4. Samples: 46573974. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:33,441][51532] Avg episode reward: [(0, '76.110'), (1, '62.220')] -[2023-10-15 18:24:33,455][52833] Updated weights for policy 0, policy_version 90840 (0.0009) -[2023-10-15 18:24:33,621][52866] Updated weights for policy 1, policy_version 91090 (0.0008) -[2023-10-15 18:24:33,991][52866] Updated weights for policy 1, policy_version 91100 (0.0007) -[2023-10-15 18:24:37,199][52833] Updated weights for policy 0, policy_version 90850 (0.0008) -[2023-10-15 18:24:37,559][52833] Updated weights for policy 0, policy_version 90860 (0.0008) -[2023-10-15 18:24:37,710][52866] Updated weights for policy 1, policy_version 91110 (0.0008) -[2023-10-15 18:24:37,925][52833] Updated weights for policy 0, policy_version 90870 (0.0008) -[2023-10-15 18:24:38,076][52866] Updated weights for policy 1, policy_version 91120 (0.0009) -[2023-10-15 18:24:38,290][52833] Updated weights for policy 0, policy_version 90880 (0.0007) -[2023-10-15 18:24:38,430][52866] Updated weights for policy 1, policy_version 91130 (0.0009) -[2023-10-15 18:24:38,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 186351616. Throughput: 0: 1800.4, 1: 1789.2. Samples: 46596214. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:38,441][51532] Avg episode reward: [(0, '76.790'), (1, '63.440')] -[2023-10-15 18:24:42,067][52833] Updated weights for policy 0, policy_version 90890 (0.0009) -[2023-10-15 18:24:42,185][52866] Updated weights for policy 1, policy_version 91140 (0.0009) -[2023-10-15 18:24:42,430][52833] Updated weights for policy 0, policy_version 90900 (0.0009) -[2023-10-15 18:24:42,556][52866] Updated weights for policy 1, policy_version 91150 (0.0008) -[2023-10-15 18:24:42,806][52833] Updated weights for policy 0, policy_version 90910 (0.0007) -[2023-10-15 18:24:42,923][52866] Updated weights for policy 1, policy_version 91160 (0.0007) -[2023-10-15 18:24:43,441][51532] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 186449920. Throughput: 0: 1794.9, 1: 1796.3. Samples: 46616334. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:43,442][51532] Avg episode reward: [(0, '79.710'), (1, '66.310')] -[2023-10-15 18:24:46,457][52833] Updated weights for policy 0, policy_version 90920 (0.0009) -[2023-10-15 18:24:46,753][52866] Updated weights for policy 1, policy_version 91170 (0.0007) -[2023-10-15 18:24:46,819][52833] Updated weights for policy 0, policy_version 90930 (0.0008) -[2023-10-15 18:24:47,114][52866] Updated weights for policy 1, policy_version 91180 (0.0007) -[2023-10-15 18:24:47,186][52833] Updated weights for policy 0, policy_version 90940 (0.0008) -[2023-10-15 18:24:47,488][52866] Updated weights for policy 1, policy_version 91190 (0.0008) -[2023-10-15 18:24:47,853][52866] Updated weights for policy 1, policy_version 91200 (0.0011) -[2023-10-15 18:24:48,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186515456. Throughput: 0: 1799.5, 1: 1790.1. Samples: 46628836. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:48,441][51532] Avg episode reward: [(0, '83.670'), (1, '63.840')] -[2023-10-15 18:24:50,871][52833] Updated weights for policy 0, policy_version 90950 (0.0008) -[2023-10-15 18:24:51,239][52833] Updated weights for policy 0, policy_version 90960 (0.0008) -[2023-10-15 18:24:51,553][52866] Updated weights for policy 1, policy_version 91210 (0.0008) -[2023-10-15 18:24:51,606][52833] Updated weights for policy 0, policy_version 90970 (0.0007) -[2023-10-15 18:24:51,925][52866] Updated weights for policy 1, policy_version 91220 (0.0008) -[2023-10-15 18:24:52,277][52866] Updated weights for policy 1, policy_version 91230 (0.0008) -[2023-10-15 18:24:53,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186580992. Throughput: 0: 1800.5, 1: 1801.3. Samples: 46648854. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:53,441][51532] Avg episode reward: [(0, '81.930'), (1, '65.670')] -[2023-10-15 18:24:55,332][52833] Updated weights for policy 0, policy_version 90980 (0.0007) -[2023-10-15 18:24:55,705][52833] Updated weights for policy 0, policy_version 90990 (0.0010) -[2023-10-15 18:24:56,075][52866] Updated weights for policy 1, policy_version 91240 (0.0008) -[2023-10-15 18:24:56,075][52833] Updated weights for policy 0, policy_version 91000 (0.0008) -[2023-10-15 18:24:56,435][52866] Updated weights for policy 1, policy_version 91250 (0.0008) -[2023-10-15 18:24:56,787][52866] Updated weights for policy 1, policy_version 91260 (0.0009) -[2023-10-15 18:24:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186646528. Throughput: 0: 1802.4, 1: 1782.5. Samples: 46670914. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) -[2023-10-15 18:24:58,441][51532] Avg episode reward: [(0, '80.480'), (1, '67.330')] -[2023-10-15 18:24:59,871][52833] Updated weights for policy 0, policy_version 91010 (0.0009) -[2023-10-15 18:25:00,240][52833] Updated weights for policy 0, policy_version 91020 (0.0010) -[2023-10-15 18:25:00,575][52866] Updated weights for policy 1, policy_version 91270 (0.0008) -[2023-10-15 18:25:00,623][52833] Updated weights for policy 0, policy_version 91030 (0.0007) -[2023-10-15 18:25:00,948][52866] Updated weights for policy 1, policy_version 91280 (0.0008) -[2023-10-15 18:25:00,985][52833] Updated weights for policy 0, policy_version 91040 (0.0009) -[2023-10-15 18:25:01,320][52866] Updated weights for policy 1, policy_version 91290 (0.0009) -[2023-10-15 18:25:03,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 186712064. Throughput: 0: 1809.2, 1: 1796.3. Samples: 46681642. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:03,442][51532] Avg episode reward: [(0, '79.480'), (1, '65.550')] -[2023-10-15 18:25:04,711][52833] Updated weights for policy 0, policy_version 91050 (0.0007) -[2023-10-15 18:25:05,085][52833] Updated weights for policy 0, policy_version 91060 (0.0008) -[2023-10-15 18:25:05,202][52866] Updated weights for policy 1, policy_version 91300 (0.0009) -[2023-10-15 18:25:05,450][52833] Updated weights for policy 0, policy_version 91070 (0.0009) -[2023-10-15 18:25:05,562][52866] Updated weights for policy 1, policy_version 91310 (0.0007) -[2023-10-15 18:25:05,933][52866] Updated weights for policy 1, policy_version 91320 (0.0007) -[2023-10-15 18:25:08,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 186777600. Throughput: 0: 1798.6, 1: 1780.8. Samples: 46702966. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:08,443][51532] Avg episode reward: [(0, '78.130'), (1, '67.100')] -[2023-10-15 18:25:09,190][52833] Updated weights for policy 0, policy_version 91080 (0.0009) -[2023-10-15 18:25:09,564][52833] Updated weights for policy 0, policy_version 91090 (0.0009) -[2023-10-15 18:25:09,670][52866] Updated weights for policy 1, policy_version 91330 (0.0009) -[2023-10-15 18:25:09,932][52833] Updated weights for policy 0, policy_version 91100 (0.0008) -[2023-10-15 18:25:10,037][52866] Updated weights for policy 1, policy_version 91340 (0.0010) -[2023-10-15 18:25:10,409][52866] Updated weights for policy 1, policy_version 91350 (0.0009) -[2023-10-15 18:25:10,778][52866] Updated weights for policy 1, policy_version 91360 (0.0008) -[2023-10-15 18:25:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 186843136. Throughput: 0: 1800.9, 1: 1777.9. Samples: 46725462. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:13,442][51532] Avg episode reward: [(0, '79.950'), (1, '66.120')] -[2023-10-15 18:25:13,603][52833] Updated weights for policy 0, policy_version 91110 (0.0009) -[2023-10-15 18:25:13,965][52833] Updated weights for policy 0, policy_version 91120 (0.0008) -[2023-10-15 18:25:14,329][52833] Updated weights for policy 0, policy_version 91130 (0.0009) -[2023-10-15 18:25:14,551][52866] Updated weights for policy 1, policy_version 91370 (0.0008) -[2023-10-15 18:25:14,917][52866] Updated weights for policy 1, policy_version 91380 (0.0008) -[2023-10-15 18:25:15,282][52866] Updated weights for policy 1, policy_version 91390 (0.0007) -[2023-10-15 18:25:18,034][52833] Updated weights for policy 0, policy_version 91140 (0.0008) -[2023-10-15 18:25:18,401][52833] Updated weights for policy 0, policy_version 91150 (0.0009) -[2023-10-15 18:25:18,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 186908672. Throughput: 0: 1806.0, 1: 1779.6. Samples: 46735330. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:18,442][51532] Avg episode reward: [(0, '81.900'), (1, '68.550')] -[2023-10-15 18:25:18,777][52833] Updated weights for policy 0, policy_version 91160 (0.0007) -[2023-10-15 18:25:19,197][52866] Updated weights for policy 1, policy_version 91400 (0.0007) -[2023-10-15 18:25:19,574][52866] Updated weights for policy 1, policy_version 91410 (0.0007) -[2023-10-15 18:25:19,934][52866] Updated weights for policy 1, policy_version 91420 (0.0008) -[2023-10-15 18:25:22,518][52833] Updated weights for policy 0, policy_version 91170 (0.0007) -[2023-10-15 18:25:22,896][52833] Updated weights for policy 0, policy_version 91180 (0.0009) -[2023-10-15 18:25:23,278][52833] Updated weights for policy 0, policy_version 91190 (0.0010) -[2023-10-15 18:25:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 186974208. Throughput: 0: 1810.7, 1: 1776.1. Samples: 46757622. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:23,442][51532] Avg episode reward: [(0, '82.780'), (1, '66.080')] -[2023-10-15 18:25:23,622][52866] Updated weights for policy 1, policy_version 91430 (0.0008) -[2023-10-15 18:25:23,645][52833] Updated weights for policy 0, policy_version 91200 (0.0009) -[2023-10-15 18:25:23,991][52866] Updated weights for policy 1, policy_version 91440 (0.0009) -[2023-10-15 18:25:24,365][52866] Updated weights for policy 1, policy_version 91450 (0.0008) -[2023-10-15 18:25:27,325][52833] Updated weights for policy 0, policy_version 91210 (0.0009) -[2023-10-15 18:25:27,689][52833] Updated weights for policy 0, policy_version 91220 (0.0008) -[2023-10-15 18:25:28,060][52833] Updated weights for policy 0, policy_version 91230 (0.0008) -[2023-10-15 18:25:28,175][52866] Updated weights for policy 1, policy_version 91460 (0.0007) -[2023-10-15 18:25:28,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 187072512. Throughput: 0: 1816.1, 1: 1797.4. Samples: 46778942. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:28,442][51532] Avg episode reward: [(0, '84.680'), (1, '66.960')] -[2023-10-15 18:25:28,527][52866] Updated weights for policy 1, policy_version 91470 (0.0010) -[2023-10-15 18:25:28,887][52866] Updated weights for policy 1, policy_version 91480 (0.0007) -[2023-10-15 18:25:31,858][52833] Updated weights for policy 0, policy_version 91240 (0.0009) -[2023-10-15 18:25:32,225][52833] Updated weights for policy 0, policy_version 91250 (0.0008) -[2023-10-15 18:25:32,592][52833] Updated weights for policy 0, policy_version 91260 (0.0007) -[2023-10-15 18:25:32,683][52866] Updated weights for policy 1, policy_version 91490 (0.0007) -[2023-10-15 18:25:33,048][52866] Updated weights for policy 1, policy_version 91500 (0.0008) -[2023-10-15 18:25:33,410][52866] Updated weights for policy 1, policy_version 91510 (0.0008) -[2023-10-15 18:25:33,441][51532] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 187138048. Throughput: 0: 1805.4, 1: 1773.6. Samples: 46789892. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:33,442][51532] Avg episode reward: [(0, '83.430'), (1, '65.900')] -[2023-10-15 18:25:33,778][52866] Updated weights for policy 1, policy_version 91520 (0.0009) -[2023-10-15 18:25:36,145][52833] Updated weights for policy 0, policy_version 91270 (0.0008) -[2023-10-15 18:25:36,503][52833] Updated weights for policy 0, policy_version 91280 (0.0010) -[2023-10-15 18:25:36,871][52833] Updated weights for policy 0, policy_version 91290 (0.0010) -[2023-10-15 18:25:37,525][52866] Updated weights for policy 1, policy_version 91530 (0.0010) -[2023-10-15 18:25:37,884][52866] Updated weights for policy 1, policy_version 91540 (0.0009) -[2023-10-15 18:25:38,253][52866] Updated weights for policy 1, policy_version 91550 (0.0010) -[2023-10-15 18:25:38,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 187236352. Throughput: 0: 1814.3, 1: 1796.2. Samples: 46811324. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:38,442][51532] Avg episode reward: [(0, '78.950'), (1, '68.040')] -[2023-10-15 18:25:40,602][52833] Updated weights for policy 0, policy_version 91300 (0.0010) -[2023-10-15 18:25:40,969][52833] Updated weights for policy 0, policy_version 91310 (0.0009) -[2023-10-15 18:25:41,352][52833] Updated weights for policy 0, policy_version 91320 (0.0008) -[2023-10-15 18:25:41,962][52866] Updated weights for policy 1, policy_version 91560 (0.0011) -[2023-10-15 18:25:42,324][52866] Updated weights for policy 1, policy_version 91570 (0.0010) -[2023-10-15 18:25:42,684][52866] Updated weights for policy 1, policy_version 91580 (0.0007) -[2023-10-15 18:25:43,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 187301888. Throughput: 0: 1801.9, 1: 1779.8. Samples: 46832092. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:43,442][51532] Avg episode reward: [(0, '79.640'), (1, '69.060')] -[2023-10-15 18:25:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000091328_93519872.pth... -[2023-10-15 18:25:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000091584_93782016.pth... -[2023-10-15 18:25:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000089888_92045312.pth -[2023-10-15 18:25:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000089632_91783168.pth -[2023-10-15 18:25:45,048][52833] Updated weights for policy 0, policy_version 91330 (0.0011) -[2023-10-15 18:25:45,424][52833] Updated weights for policy 0, policy_version 91340 (0.0011) -[2023-10-15 18:25:45,785][52833] Updated weights for policy 0, policy_version 91350 (0.0007) -[2023-10-15 18:25:46,151][52833] Updated weights for policy 0, policy_version 91360 (0.0009) -[2023-10-15 18:25:46,467][52866] Updated weights for policy 1, policy_version 91590 (0.0007) -[2023-10-15 18:25:46,828][52866] Updated weights for policy 1, policy_version 91600 (0.0007) -[2023-10-15 18:25:47,190][52866] Updated weights for policy 1, policy_version 91610 (0.0009) -[2023-10-15 18:25:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 187367424. Throughput: 0: 1804.0, 1: 1798.0. Samples: 46843730. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:48,441][51532] Avg episode reward: [(0, '75.140'), (1, '70.280')] -[2023-10-15 18:25:50,079][52833] Updated weights for policy 0, policy_version 91370 (0.0007) -[2023-10-15 18:25:50,455][52833] Updated weights for policy 0, policy_version 91380 (0.0007) -[2023-10-15 18:25:50,824][52833] Updated weights for policy 0, policy_version 91390 (0.0008) -[2023-10-15 18:25:51,036][52866] Updated weights for policy 1, policy_version 91620 (0.0009) -[2023-10-15 18:25:51,398][52866] Updated weights for policy 1, policy_version 91630 (0.0009) -[2023-10-15 18:25:51,765][52866] Updated weights for policy 1, policy_version 91640 (0.0009) -[2023-10-15 18:25:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 187432960. Throughput: 0: 1801.3, 1: 1782.4. Samples: 46864234. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:53,442][51532] Avg episode reward: [(0, '75.380'), (1, '69.590')] -[2023-10-15 18:25:54,674][52833] Updated weights for policy 0, policy_version 91400 (0.0008) -[2023-10-15 18:25:55,050][52833] Updated weights for policy 0, policy_version 91410 (0.0007) -[2023-10-15 18:25:55,431][52833] Updated weights for policy 0, policy_version 91420 (0.0008) -[2023-10-15 18:25:55,533][52866] Updated weights for policy 1, policy_version 91650 (0.0010) -[2023-10-15 18:25:55,891][52866] Updated weights for policy 1, policy_version 91660 (0.0007) -[2023-10-15 18:25:56,253][52866] Updated weights for policy 1, policy_version 91670 (0.0008) -[2023-10-15 18:25:56,622][52866] Updated weights for policy 1, policy_version 91680 (0.0008) -[2023-10-15 18:25:58,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 187498496. Throughput: 0: 1795.9, 1: 1784.0. Samples: 46886556. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) -[2023-10-15 18:25:58,442][51532] Avg episode reward: [(0, '78.170'), (1, '68.160')] -[2023-10-15 18:25:59,112][52833] Updated weights for policy 0, policy_version 91430 (0.0007) -[2023-10-15 18:25:59,487][52833] Updated weights for policy 0, policy_version 91440 (0.0008) -[2023-10-15 18:25:59,849][52833] Updated weights for policy 0, policy_version 91450 (0.0009) -[2023-10-15 18:26:00,505][52866] Updated weights for policy 1, policy_version 91690 (0.0008) -[2023-10-15 18:26:00,864][52866] Updated weights for policy 1, policy_version 91700 (0.0009) -[2023-10-15 18:26:01,238][52866] Updated weights for policy 1, policy_version 91710 (0.0007) -[2023-10-15 18:26:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 187564032. Throughput: 0: 1789.9, 1: 1797.2. Samples: 46896754. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:03,442][51532] Avg episode reward: [(0, '79.950'), (1, '67.640')] -[2023-10-15 18:26:03,625][52833] Updated weights for policy 0, policy_version 91460 (0.0008) -[2023-10-15 18:26:03,997][52833] Updated weights for policy 0, policy_version 91470 (0.0009) -[2023-10-15 18:26:04,373][52833] Updated weights for policy 0, policy_version 91480 (0.0010) -[2023-10-15 18:26:04,935][52866] Updated weights for policy 1, policy_version 91720 (0.0008) -[2023-10-15 18:26:05,302][52866] Updated weights for policy 1, policy_version 91730 (0.0010) -[2023-10-15 18:26:05,676][52866] Updated weights for policy 1, policy_version 91740 (0.0008) -[2023-10-15 18:26:08,307][52833] Updated weights for policy 0, policy_version 91490 (0.0009) -[2023-10-15 18:26:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 187629568. Throughput: 0: 1784.6, 1: 1794.6. Samples: 46918688. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:08,442][51532] Avg episode reward: [(0, '82.720'), (1, '65.460')] -[2023-10-15 18:26:08,679][52833] Updated weights for policy 0, policy_version 91500 (0.0007) -[2023-10-15 18:26:09,042][52833] Updated weights for policy 0, policy_version 91510 (0.0010) -[2023-10-15 18:26:09,298][52866] Updated weights for policy 1, policy_version 91750 (0.0010) -[2023-10-15 18:26:09,418][52833] Updated weights for policy 0, policy_version 91520 (0.0009) -[2023-10-15 18:26:09,663][52866] Updated weights for policy 1, policy_version 91760 (0.0007) -[2023-10-15 18:26:10,026][52866] Updated weights for policy 1, policy_version 91770 (0.0007) -[2023-10-15 18:26:13,059][52833] Updated weights for policy 0, policy_version 91530 (0.0007) -[2023-10-15 18:26:13,432][52833] Updated weights for policy 0, policy_version 91540 (0.0007) -[2023-10-15 18:26:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 187695104. Throughput: 0: 1807.1, 1: 1793.4. Samples: 46940964. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:13,442][51532] Avg episode reward: [(0, '83.610'), (1, '63.900')] -[2023-10-15 18:26:13,801][52833] Updated weights for policy 0, policy_version 91550 (0.0009) -[2023-10-15 18:26:13,888][52866] Updated weights for policy 1, policy_version 91780 (0.0007) -[2023-10-15 18:26:14,249][52866] Updated weights for policy 1, policy_version 91790 (0.0008) -[2023-10-15 18:26:14,623][52866] Updated weights for policy 1, policy_version 91800 (0.0009) -[2023-10-15 18:26:17,496][52833] Updated weights for policy 0, policy_version 91560 (0.0009) -[2023-10-15 18:26:17,869][52833] Updated weights for policy 0, policy_version 91570 (0.0009) -[2023-10-15 18:26:18,208][52866] Updated weights for policy 1, policy_version 91810 (0.0007) -[2023-10-15 18:26:18,246][52833] Updated weights for policy 0, policy_version 91580 (0.0008) -[2023-10-15 18:26:18,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 187793408. Throughput: 0: 1788.8, 1: 1796.1. Samples: 46951212. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:18,441][51532] Avg episode reward: [(0, '84.600'), (1, '63.420')] -[2023-10-15 18:26:18,570][52866] Updated weights for policy 1, policy_version 91820 (0.0009) -[2023-10-15 18:26:18,944][52866] Updated weights for policy 1, policy_version 91830 (0.0009) -[2023-10-15 18:26:19,309][52866] Updated weights for policy 1, policy_version 91840 (0.0008) -[2023-10-15 18:26:22,099][52833] Updated weights for policy 0, policy_version 91590 (0.0010) -[2023-10-15 18:26:22,460][52833] Updated weights for policy 0, policy_version 91600 (0.0010) -[2023-10-15 18:26:22,827][52833] Updated weights for policy 0, policy_version 91610 (0.0007) -[2023-10-15 18:26:22,893][52866] Updated weights for policy 1, policy_version 91850 (0.0008) -[2023-10-15 18:26:23,273][52866] Updated weights for policy 1, policy_version 91860 (0.0009) -[2023-10-15 18:26:23,441][51532] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14329.1). Total num frames: 187858944. Throughput: 0: 1809.4, 1: 1799.1. Samples: 46973706. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:23,441][51532] Avg episode reward: [(0, '83.130'), (1, '63.590')] -[2023-10-15 18:26:23,640][52866] Updated weights for policy 1, policy_version 91870 (0.0007) -[2023-10-15 18:26:26,495][52833] Updated weights for policy 0, policy_version 91620 (0.0010) -[2023-10-15 18:26:26,868][52833] Updated weights for policy 0, policy_version 91630 (0.0009) -[2023-10-15 18:26:27,241][52833] Updated weights for policy 0, policy_version 91640 (0.0009) -[2023-10-15 18:26:27,484][52866] Updated weights for policy 1, policy_version 91880 (0.0008) -[2023-10-15 18:26:27,842][52866] Updated weights for policy 1, policy_version 91890 (0.0007) -[2023-10-15 18:26:28,216][52866] Updated weights for policy 1, policy_version 91900 (0.0007) -[2023-10-15 18:26:28,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 187957248. Throughput: 0: 1783.0, 1: 1812.2. Samples: 46993876. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:28,442][51532] Avg episode reward: [(0, '82.230'), (1, '61.880')] -[2023-10-15 18:26:30,988][52833] Updated weights for policy 0, policy_version 91650 (0.0009) -[2023-10-15 18:26:31,355][52833] Updated weights for policy 0, policy_version 91660 (0.0010) -[2023-10-15 18:26:31,720][52833] Updated weights for policy 0, policy_version 91670 (0.0007) -[2023-10-15 18:26:31,844][52866] Updated weights for policy 1, policy_version 91910 (0.0007) -[2023-10-15 18:26:32,085][52833] Updated weights for policy 0, policy_version 91680 (0.0007) -[2023-10-15 18:26:32,206][52866] Updated weights for policy 1, policy_version 91920 (0.0008) -[2023-10-15 18:26:32,572][52866] Updated weights for policy 1, policy_version 91930 (0.0008) -[2023-10-15 18:26:33,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 188022784. Throughput: 0: 1807.2, 1: 1802.8. Samples: 47006184. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:33,442][51532] Avg episode reward: [(0, '83.420'), (1, '58.230')] -[2023-10-15 18:26:35,804][52833] Updated weights for policy 0, policy_version 91690 (0.0010) -[2023-10-15 18:26:36,171][52833] Updated weights for policy 0, policy_version 91700 (0.0012) -[2023-10-15 18:26:36,293][52866] Updated weights for policy 1, policy_version 91940 (0.0009) -[2023-10-15 18:26:36,542][52833] Updated weights for policy 0, policy_version 91710 (0.0008) -[2023-10-15 18:26:36,651][52866] Updated weights for policy 1, policy_version 91950 (0.0009) -[2023-10-15 18:26:37,014][52866] Updated weights for policy 1, policy_version 91960 (0.0008) -[2023-10-15 18:26:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188088320. Throughput: 0: 1783.0, 1: 1813.4. Samples: 47026072. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:38,442][51532] Avg episode reward: [(0, '81.210'), (1, '57.280')] -[2023-10-15 18:26:40,447][52833] Updated weights for policy 0, policy_version 91720 (0.0007) -[2023-10-15 18:26:40,633][52866] Updated weights for policy 1, policy_version 91970 (0.0007) -[2023-10-15 18:26:40,807][52833] Updated weights for policy 0, policy_version 91730 (0.0008) -[2023-10-15 18:26:40,994][52866] Updated weights for policy 1, policy_version 91980 (0.0007) -[2023-10-15 18:26:41,179][52833] Updated weights for policy 0, policy_version 91740 (0.0008) -[2023-10-15 18:26:41,359][52866] Updated weights for policy 1, policy_version 91990 (0.0009) -[2023-10-15 18:26:41,727][52866] Updated weights for policy 1, policy_version 92000 (0.0009) -[2023-10-15 18:26:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 188153856. Throughput: 0: 1780.5, 1: 1803.2. Samples: 47047824. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:43,442][51532] Avg episode reward: [(0, '81.160'), (1, '57.400')] -[2023-10-15 18:26:45,009][52833] Updated weights for policy 0, policy_version 91750 (0.0009) -[2023-10-15 18:26:45,373][52833] Updated weights for policy 0, policy_version 91760 (0.0009) -[2023-10-15 18:26:45,745][52833] Updated weights for policy 0, policy_version 91770 (0.0007) -[2023-10-15 18:26:45,749][52866] Updated weights for policy 1, policy_version 92010 (0.0009) -[2023-10-15 18:26:46,119][52866] Updated weights for policy 1, policy_version 92020 (0.0009) -[2023-10-15 18:26:46,485][52866] Updated weights for policy 1, policy_version 92030 (0.0008) -[2023-10-15 18:26:48,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188219392. Throughput: 0: 1783.9, 1: 1799.8. Samples: 47058018. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:48,441][51532] Avg episode reward: [(0, '84.080'), (1, '58.160')] -[2023-10-15 18:26:49,591][52833] Updated weights for policy 0, policy_version 91780 (0.0009) -[2023-10-15 18:26:49,957][52833] Updated weights for policy 0, policy_version 91790 (0.0008) -[2023-10-15 18:26:50,327][52833] Updated weights for policy 0, policy_version 91800 (0.0007) -[2023-10-15 18:26:50,338][52866] Updated weights for policy 1, policy_version 92040 (0.0007) -[2023-10-15 18:26:50,698][52866] Updated weights for policy 1, policy_version 92050 (0.0008) -[2023-10-15 18:26:51,066][52866] Updated weights for policy 1, policy_version 92060 (0.0008) -[2023-10-15 18:26:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188284928. Throughput: 0: 1775.2, 1: 1790.6. Samples: 47079150. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:53,442][51532] Avg episode reward: [(0, '85.200'), (1, '55.810')] -[2023-10-15 18:26:54,031][52833] Updated weights for policy 0, policy_version 91810 (0.0007) -[2023-10-15 18:26:54,401][52833] Updated weights for policy 0, policy_version 91820 (0.0008) -[2023-10-15 18:26:54,700][52866] Updated weights for policy 1, policy_version 92070 (0.0010) -[2023-10-15 18:26:54,781][52833] Updated weights for policy 0, policy_version 91830 (0.0007) -[2023-10-15 18:26:55,064][52866] Updated weights for policy 1, policy_version 92080 (0.0007) -[2023-10-15 18:26:55,139][52833] Updated weights for policy 0, policy_version 91840 (0.0008) -[2023-10-15 18:26:55,428][52866] Updated weights for policy 1, policy_version 92090 (0.0007) -[2023-10-15 18:26:58,441][51532] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 188350464. Throughput: 0: 1784.1, 1: 1802.1. Samples: 47102344. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-15 18:26:58,442][51532] Avg episode reward: [(0, '86.620'), (1, '60.000')] -[2023-10-15 18:26:58,741][52833] Updated weights for policy 0, policy_version 91850 (0.0009) -[2023-10-15 18:26:59,109][52866] Updated weights for policy 1, policy_version 92100 (0.0007) -[2023-10-15 18:26:59,120][52833] Updated weights for policy 0, policy_version 91860 (0.0009) -[2023-10-15 18:26:59,474][52866] Updated weights for policy 1, policy_version 92110 (0.0007) -[2023-10-15 18:26:59,485][52833] Updated weights for policy 0, policy_version 91870 (0.0007) -[2023-10-15 18:26:59,839][52866] Updated weights for policy 1, policy_version 92120 (0.0008) -[2023-10-15 18:27:03,431][52833] Updated weights for policy 0, policy_version 91880 (0.0008) -[2023-10-15 18:27:03,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 188416000. Throughput: 0: 1776.4, 1: 1799.9. Samples: 47112146. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:03,441][51532] Avg episode reward: [(0, '83.520'), (1, '57.870')] -[2023-10-15 18:27:03,728][52866] Updated weights for policy 1, policy_version 92130 (0.0009) -[2023-10-15 18:27:03,788][52833] Updated weights for policy 0, policy_version 91890 (0.0008) -[2023-10-15 18:27:04,097][52866] Updated weights for policy 1, policy_version 92140 (0.0008) -[2023-10-15 18:27:04,163][52833] Updated weights for policy 0, policy_version 91900 (0.0008) -[2023-10-15 18:27:04,463][52866] Updated weights for policy 1, policy_version 92150 (0.0009) -[2023-10-15 18:27:04,832][52866] Updated weights for policy 1, policy_version 92160 (0.0010) -[2023-10-15 18:27:07,948][52833] Updated weights for policy 0, policy_version 91910 (0.0008) -[2023-10-15 18:27:08,323][52833] Updated weights for policy 0, policy_version 91920 (0.0007) -[2023-10-15 18:27:08,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 188481536. Throughput: 0: 1779.9, 1: 1788.8. Samples: 47134298. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:08,441][51532] Avg episode reward: [(0, '79.190'), (1, '58.060')] -[2023-10-15 18:27:08,621][52866] Updated weights for policy 1, policy_version 92170 (0.0007) -[2023-10-15 18:27:08,680][52833] Updated weights for policy 0, policy_version 91930 (0.0007) -[2023-10-15 18:27:08,974][52866] Updated weights for policy 1, policy_version 92180 (0.0007) -[2023-10-15 18:27:09,336][52866] Updated weights for policy 1, policy_version 92190 (0.0008) -[2023-10-15 18:27:12,552][52833] Updated weights for policy 0, policy_version 91940 (0.0008) -[2023-10-15 18:27:12,911][52833] Updated weights for policy 0, policy_version 91950 (0.0007) -[2023-10-15 18:27:13,183][52866] Updated weights for policy 1, policy_version 92200 (0.0009) -[2023-10-15 18:27:13,282][52833] Updated weights for policy 0, policy_version 91960 (0.0007) -[2023-10-15 18:27:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 188547072. Throughput: 0: 1801.5, 1: 1804.5. Samples: 47156146. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:13,441][51532] Avg episode reward: [(0, '78.540'), (1, '52.680')] -[2023-10-15 18:27:13,542][52866] Updated weights for policy 1, policy_version 92210 (0.0007) -[2023-10-15 18:27:13,910][52866] Updated weights for policy 1, policy_version 92220 (0.0010) -[2023-10-15 18:27:17,056][52833] Updated weights for policy 0, policy_version 91970 (0.0007) -[2023-10-15 18:27:17,430][52833] Updated weights for policy 0, policy_version 91980 (0.0007) -[2023-10-15 18:27:17,619][52866] Updated weights for policy 1, policy_version 92230 (0.0009) -[2023-10-15 18:27:17,791][52833] Updated weights for policy 0, policy_version 91990 (0.0007) -[2023-10-15 18:27:17,990][52866] Updated weights for policy 1, policy_version 92240 (0.0008) -[2023-10-15 18:27:18,150][52833] Updated weights for policy 0, policy_version 92000 (0.0007) -[2023-10-15 18:27:18,351][52866] Updated weights for policy 1, policy_version 92250 (0.0008) -[2023-10-15 18:27:18,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 188645376. Throughput: 0: 1781.3, 1: 1785.6. Samples: 47166698. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:18,442][51532] Avg episode reward: [(0, '74.920'), (1, '53.240')] -[2023-10-15 18:27:21,748][52833] Updated weights for policy 0, policy_version 92010 (0.0007) -[2023-10-15 18:27:22,118][52833] Updated weights for policy 0, policy_version 92020 (0.0007) -[2023-10-15 18:27:22,155][52866] Updated weights for policy 1, policy_version 92260 (0.0008) -[2023-10-15 18:27:22,483][52833] Updated weights for policy 0, policy_version 92030 (0.0007) -[2023-10-15 18:27:22,520][52866] Updated weights for policy 1, policy_version 92270 (0.0007) -[2023-10-15 18:27:22,890][52866] Updated weights for policy 1, policy_version 92280 (0.0010) -[2023-10-15 18:27:23,441][51532] Fps is (10 sec: 19660.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 188743680. Throughput: 0: 1806.8, 1: 1803.4. Samples: 47188528. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:23,442][51532] Avg episode reward: [(0, '76.990'), (1, '53.070')] -[2023-10-15 18:27:26,406][52833] Updated weights for policy 0, policy_version 92040 (0.0008) -[2023-10-15 18:27:26,658][52866] Updated weights for policy 1, policy_version 92290 (0.0007) -[2023-10-15 18:27:26,775][52833] Updated weights for policy 0, policy_version 92050 (0.0009) -[2023-10-15 18:27:27,019][52866] Updated weights for policy 1, policy_version 92300 (0.0008) -[2023-10-15 18:27:27,147][52833] Updated weights for policy 0, policy_version 92060 (0.0008) -[2023-10-15 18:27:27,387][52866] Updated weights for policy 1, policy_version 92310 (0.0009) -[2023-10-15 18:27:27,748][52866] Updated weights for policy 1, policy_version 92320 (0.0010) -[2023-10-15 18:27:28,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188809216. Throughput: 0: 1784.2, 1: 1779.2. Samples: 47208176. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:28,442][51532] Avg episode reward: [(0, '75.600'), (1, '55.990')] -[2023-10-15 18:27:30,889][52833] Updated weights for policy 0, policy_version 92070 (0.0007) -[2023-10-15 18:27:31,244][52833] Updated weights for policy 0, policy_version 92080 (0.0007) -[2023-10-15 18:27:31,444][52866] Updated weights for policy 1, policy_version 92330 (0.0008) -[2023-10-15 18:27:31,608][52833] Updated weights for policy 0, policy_version 92090 (0.0008) -[2023-10-15 18:27:31,812][52866] Updated weights for policy 1, policy_version 92340 (0.0008) -[2023-10-15 18:27:32,183][52866] Updated weights for policy 1, policy_version 92350 (0.0009) -[2023-10-15 18:27:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188874752. Throughput: 0: 1808.6, 1: 1811.2. Samples: 47220910. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:33,442][51532] Avg episode reward: [(0, '74.740'), (1, '57.290')] -[2023-10-15 18:27:35,288][52833] Updated weights for policy 0, policy_version 92100 (0.0009) -[2023-10-15 18:27:35,650][52833] Updated weights for policy 0, policy_version 92110 (0.0008) -[2023-10-15 18:27:36,023][52833] Updated weights for policy 0, policy_version 92120 (0.0008) -[2023-10-15 18:27:36,142][52866] Updated weights for policy 1, policy_version 92360 (0.0008) -[2023-10-15 18:27:36,509][52866] Updated weights for policy 1, policy_version 92370 (0.0008) -[2023-10-15 18:27:36,883][52866] Updated weights for policy 1, policy_version 92380 (0.0008) -[2023-10-15 18:27:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188940288. Throughput: 0: 1794.8, 1: 1795.8. Samples: 47240726. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:38,442][51532] Avg episode reward: [(0, '76.440'), (1, '54.250')] -[2023-10-15 18:27:39,519][52833] Updated weights for policy 0, policy_version 92130 (0.0007) -[2023-10-15 18:27:39,892][52833] Updated weights for policy 0, policy_version 92140 (0.0007) -[2023-10-15 18:27:40,262][52833] Updated weights for policy 0, policy_version 92150 (0.0007) -[2023-10-15 18:27:40,556][52866] Updated weights for policy 1, policy_version 92390 (0.0008) -[2023-10-15 18:27:40,628][52833] Updated weights for policy 0, policy_version 92160 (0.0007) -[2023-10-15 18:27:40,923][52866] Updated weights for policy 1, policy_version 92400 (0.0007) -[2023-10-15 18:27:41,289][52866] Updated weights for policy 1, policy_version 92410 (0.0010) -[2023-10-15 18:27:43,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 189005824. Throughput: 0: 1791.7, 1: 1785.2. Samples: 47263304. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:43,442][51532] Avg episode reward: [(0, '76.110'), (1, '52.950')] -[2023-10-15 18:27:43,453][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000092160_94371840.pth... -[2023-10-15 18:27:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000092416_94633984.pth... -[2023-10-15 18:27:43,488][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000090752_92930048.pth -[2023-10-15 18:27:43,492][52518] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/milestones/checkpoint_000092416_94633984.pth -[2023-10-15 18:27:43,493][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000090496_92667904.pth -[2023-10-15 18:27:43,499][52410] Saving a milestone ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/milestones/checkpoint_000092160_94371840.pth -[2023-10-15 18:27:44,473][52833] Updated weights for policy 0, policy_version 92170 (0.0009) -[2023-10-15 18:27:44,838][52833] Updated weights for policy 0, policy_version 92180 (0.0008) -[2023-10-15 18:27:44,975][52866] Updated weights for policy 1, policy_version 92420 (0.0009) -[2023-10-15 18:27:45,217][52833] Updated weights for policy 0, policy_version 92190 (0.0008) -[2023-10-15 18:27:45,339][52866] Updated weights for policy 1, policy_version 92430 (0.0008) -[2023-10-15 18:27:45,713][52866] Updated weights for policy 1, policy_version 92440 (0.0010) -[2023-10-15 18:27:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 189071360. Throughput: 0: 1791.1, 1: 1790.8. Samples: 47273330. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:48,442][51532] Avg episode reward: [(0, '75.430'), (1, '52.230')] -[2023-10-15 18:27:49,193][52833] Updated weights for policy 0, policy_version 92200 (0.0007) -[2023-10-15 18:27:49,520][52866] Updated weights for policy 1, policy_version 92450 (0.0010) -[2023-10-15 18:27:49,574][52833] Updated weights for policy 0, policy_version 92210 (0.0009) -[2023-10-15 18:27:49,884][52866] Updated weights for policy 1, policy_version 92460 (0.0007) -[2023-10-15 18:27:49,932][52833] Updated weights for policy 0, policy_version 92220 (0.0008) -[2023-10-15 18:27:50,255][52866] Updated weights for policy 1, policy_version 92470 (0.0007) -[2023-10-15 18:27:50,615][52866] Updated weights for policy 1, policy_version 92480 (0.0007) -[2023-10-15 18:27:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 189136896. Throughput: 0: 1788.7, 1: 1794.4. Samples: 47295536. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:53,442][51532] Avg episode reward: [(0, '73.270'), (1, '52.500')] -[2023-10-15 18:27:53,659][52833] Updated weights for policy 0, policy_version 92230 (0.0010) -[2023-10-15 18:27:54,035][52833] Updated weights for policy 0, policy_version 92240 (0.0009) -[2023-10-15 18:27:54,286][52866] Updated weights for policy 1, policy_version 92490 (0.0008) -[2023-10-15 18:27:54,396][52833] Updated weights for policy 0, policy_version 92250 (0.0007) -[2023-10-15 18:27:54,646][52866] Updated weights for policy 1, policy_version 92500 (0.0008) -[2023-10-15 18:27:55,021][52866] Updated weights for policy 1, policy_version 92510 (0.0010) -[2023-10-15 18:27:58,082][52833] Updated weights for policy 0, policy_version 92260 (0.0009) -[2023-10-15 18:27:58,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 189202432. Throughput: 0: 1806.3, 1: 1802.4. Samples: 47318538. Policy #0 lag: (min: 8.0, avg: 33.6, max: 40.0) -[2023-10-15 18:27:58,441][51532] Avg episode reward: [(0, '74.710'), (1, '57.640')] -[2023-10-15 18:27:58,456][52833] Updated weights for policy 0, policy_version 92270 (0.0009) -[2023-10-15 18:27:58,671][52866] Updated weights for policy 1, policy_version 92520 (0.0008) -[2023-10-15 18:27:58,833][52833] Updated weights for policy 0, policy_version 92280 (0.0008) -[2023-10-15 18:27:59,033][52866] Updated weights for policy 1, policy_version 92530 (0.0008) -[2023-10-15 18:27:59,395][52866] Updated weights for policy 1, policy_version 92540 (0.0007) -[2023-10-15 18:28:02,632][52833] Updated weights for policy 0, policy_version 92290 (0.0008) -[2023-10-15 18:28:02,999][52833] Updated weights for policy 0, policy_version 92300 (0.0010) -[2023-10-15 18:28:03,099][52866] Updated weights for policy 1, policy_version 92550 (0.0008) -[2023-10-15 18:28:03,366][52833] Updated weights for policy 0, policy_version 92310 (0.0008) -[2023-10-15 18:28:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 189267968. Throughput: 0: 1793.8, 1: 1799.5. Samples: 47328394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:03,441][51532] Avg episode reward: [(0, '77.270'), (1, '58.250')] -[2023-10-15 18:28:03,467][52866] Updated weights for policy 1, policy_version 92560 (0.0009) -[2023-10-15 18:28:03,734][52833] Updated weights for policy 0, policy_version 92320 (0.0009) -[2023-10-15 18:28:03,835][52866] Updated weights for policy 1, policy_version 92570 (0.0008) -[2023-10-15 18:28:07,470][52833] Updated weights for policy 0, policy_version 92330 (0.0008) -[2023-10-15 18:28:07,586][52866] Updated weights for policy 1, policy_version 92580 (0.0010) -[2023-10-15 18:28:07,841][52833] Updated weights for policy 0, policy_version 92340 (0.0008) -[2023-10-15 18:28:07,952][52866] Updated weights for policy 1, policy_version 92590 (0.0009) -[2023-10-15 18:28:08,206][52833] Updated weights for policy 0, policy_version 92350 (0.0008) -[2023-10-15 18:28:08,310][52866] Updated weights for policy 1, policy_version 92600 (0.0007) -[2023-10-15 18:28:08,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 189366272. Throughput: 0: 1798.2, 1: 1802.4. Samples: 47350552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:08,441][51532] Avg episode reward: [(0, '76.550'), (1, '57.810')] -[2023-10-15 18:28:12,009][52833] Updated weights for policy 0, policy_version 92360 (0.0009) -[2023-10-15 18:28:12,096][52866] Updated weights for policy 1, policy_version 92610 (0.0008) -[2023-10-15 18:28:12,384][52833] Updated weights for policy 0, policy_version 92370 (0.0008) -[2023-10-15 18:28:12,457][52866] Updated weights for policy 1, policy_version 92620 (0.0008) -[2023-10-15 18:28:12,753][52833] Updated weights for policy 0, policy_version 92380 (0.0008) -[2023-10-15 18:28:12,820][52866] Updated weights for policy 1, policy_version 92630 (0.0007) -[2023-10-15 18:28:13,188][52866] Updated weights for policy 1, policy_version 92640 (0.0010) -[2023-10-15 18:28:13,441][51532] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 189464576. Throughput: 0: 1792.5, 1: 1810.9. Samples: 47370330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:13,442][51532] Avg episode reward: [(0, '74.310'), (1, '60.780')] -[2023-10-15 18:28:16,396][52833] Updated weights for policy 0, policy_version 92390 (0.0009) -[2023-10-15 18:28:16,762][52833] Updated weights for policy 0, policy_version 92400 (0.0008) -[2023-10-15 18:28:17,134][52833] Updated weights for policy 0, policy_version 92410 (0.0007) -[2023-10-15 18:28:17,134][52866] Updated weights for policy 1, policy_version 92650 (0.0007) -[2023-10-15 18:28:17,497][52866] Updated weights for policy 1, policy_version 92660 (0.0007) -[2023-10-15 18:28:17,851][52866] Updated weights for policy 1, policy_version 92670 (0.0007) -[2023-10-15 18:28:18,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 189530112. Throughput: 0: 1801.6, 1: 1793.3. Samples: 47382680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:18,442][51532] Avg episode reward: [(0, '74.780'), (1, '60.050')] -[2023-10-15 18:28:20,898][52833] Updated weights for policy 0, policy_version 92420 (0.0008) -[2023-10-15 18:28:21,275][52833] Updated weights for policy 0, policy_version 92430 (0.0010) -[2023-10-15 18:28:21,474][52866] Updated weights for policy 1, policy_version 92680 (0.0008) -[2023-10-15 18:28:21,635][52833] Updated weights for policy 0, policy_version 92440 (0.0009) -[2023-10-15 18:28:21,836][52866] Updated weights for policy 1, policy_version 92690 (0.0008) -[2023-10-15 18:28:22,206][52866] Updated weights for policy 1, policy_version 92700 (0.0009) -[2023-10-15 18:28:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 189595648. Throughput: 0: 1797.3, 1: 1803.3. Samples: 47402754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:23,441][51532] Avg episode reward: [(0, '75.220'), (1, '60.790')] -[2023-10-15 18:28:25,187][52833] Updated weights for policy 0, policy_version 92450 (0.0009) -[2023-10-15 18:28:25,562][52833] Updated weights for policy 0, policy_version 92460 (0.0009) -[2023-10-15 18:28:25,929][52833] Updated weights for policy 0, policy_version 92470 (0.0008) -[2023-10-15 18:28:25,997][52866] Updated weights for policy 1, policy_version 92710 (0.0007) -[2023-10-15 18:28:26,306][52833] Updated weights for policy 0, policy_version 92480 (0.0007) -[2023-10-15 18:28:26,365][52866] Updated weights for policy 1, policy_version 92720 (0.0008) -[2023-10-15 18:28:26,735][52866] Updated weights for policy 1, policy_version 92730 (0.0008) -[2023-10-15 18:28:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 189661184. Throughput: 0: 1797.9, 1: 1795.9. Samples: 47425024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:28,442][51532] Avg episode reward: [(0, '75.370'), (1, '59.970')] -[2023-10-15 18:28:30,246][52833] Updated weights for policy 0, policy_version 92490 (0.0008) -[2023-10-15 18:28:30,391][52866] Updated weights for policy 1, policy_version 92740 (0.0008) -[2023-10-15 18:28:30,624][52833] Updated weights for policy 0, policy_version 92500 (0.0007) -[2023-10-15 18:28:30,756][52866] Updated weights for policy 1, policy_version 92750 (0.0008) -[2023-10-15 18:28:30,986][52833] Updated weights for policy 0, policy_version 92510 (0.0008) -[2023-10-15 18:28:31,115][52866] Updated weights for policy 1, policy_version 92760 (0.0008) -[2023-10-15 18:28:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 189726720. Throughput: 0: 1803.9, 1: 1801.6. Samples: 47435582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:33,442][51532] Avg episode reward: [(0, '74.940'), (1, '62.130')] -[2023-10-15 18:28:34,767][52833] Updated weights for policy 0, policy_version 92520 (0.0008) -[2023-10-15 18:28:34,867][52866] Updated weights for policy 1, policy_version 92770 (0.0009) -[2023-10-15 18:28:35,136][52833] Updated weights for policy 0, policy_version 92530 (0.0009) -[2023-10-15 18:28:35,244][52866] Updated weights for policy 1, policy_version 92780 (0.0008) -[2023-10-15 18:28:35,506][52833] Updated weights for policy 0, policy_version 92540 (0.0009) -[2023-10-15 18:28:35,615][52866] Updated weights for policy 1, policy_version 92790 (0.0007) -[2023-10-15 18:28:35,984][52866] Updated weights for policy 1, policy_version 92800 (0.0011) -[2023-10-15 18:28:38,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 189792256. Throughput: 0: 1799.1, 1: 1790.1. Samples: 47457050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:38,442][51532] Avg episode reward: [(0, '73.290'), (1, '62.160')] -[2023-10-15 18:28:39,230][52833] Updated weights for policy 0, policy_version 92550 (0.0009) -[2023-10-15 18:28:39,597][52833] Updated weights for policy 0, policy_version 92560 (0.0009) -[2023-10-15 18:28:39,702][52866] Updated weights for policy 1, policy_version 92810 (0.0009) -[2023-10-15 18:28:39,964][52833] Updated weights for policy 0, policy_version 92570 (0.0007) -[2023-10-15 18:28:40,066][52866] Updated weights for policy 1, policy_version 92820 (0.0009) -[2023-10-15 18:28:40,430][52866] Updated weights for policy 1, policy_version 92830 (0.0008) -[2023-10-15 18:28:43,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 189857792. Throughput: 0: 1794.3, 1: 1782.4. Samples: 47479490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:43,442][51532] Avg episode reward: [(0, '71.170'), (1, '64.820')] -[2023-10-15 18:28:43,726][52833] Updated weights for policy 0, policy_version 92580 (0.0009) -[2023-10-15 18:28:44,087][52833] Updated weights for policy 0, policy_version 92590 (0.0009) -[2023-10-15 18:28:44,220][52866] Updated weights for policy 1, policy_version 92840 (0.0009) -[2023-10-15 18:28:44,444][52833] Updated weights for policy 0, policy_version 92600 (0.0009) -[2023-10-15 18:28:44,598][52866] Updated weights for policy 1, policy_version 92850 (0.0009) -[2023-10-15 18:28:44,948][52866] Updated weights for policy 1, policy_version 92860 (0.0008) -[2023-10-15 18:28:48,117][52833] Updated weights for policy 0, policy_version 92610 (0.0009) -[2023-10-15 18:28:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 189923328. Throughput: 0: 1789.9, 1: 1779.2. Samples: 47489000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:48,441][51532] Avg episode reward: [(0, '71.650'), (1, '61.130')] -[2023-10-15 18:28:48,485][52833] Updated weights for policy 0, policy_version 92620 (0.0008) -[2023-10-15 18:28:48,847][52833] Updated weights for policy 0, policy_version 92630 (0.0009) -[2023-10-15 18:28:48,908][52866] Updated weights for policy 1, policy_version 92870 (0.0009) -[2023-10-15 18:28:49,213][52833] Updated weights for policy 0, policy_version 92640 (0.0008) -[2023-10-15 18:28:49,269][52866] Updated weights for policy 1, policy_version 92880 (0.0008) -[2023-10-15 18:28:49,641][52866] Updated weights for policy 1, policy_version 92890 (0.0008) -[2023-10-15 18:28:52,948][52833] Updated weights for policy 0, policy_version 92650 (0.0009) -[2023-10-15 18:28:53,279][52866] Updated weights for policy 1, policy_version 92900 (0.0008) -[2023-10-15 18:28:53,320][52833] Updated weights for policy 0, policy_version 92660 (0.0008) -[2023-10-15 18:28:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 189988864. Throughput: 0: 1794.9, 1: 1777.9. Samples: 47511328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:53,442][51532] Avg episode reward: [(0, '73.750'), (1, '57.640')] -[2023-10-15 18:28:53,650][52866] Updated weights for policy 1, policy_version 92910 (0.0008) -[2023-10-15 18:28:53,684][52833] Updated weights for policy 0, policy_version 92670 (0.0008) -[2023-10-15 18:28:54,011][52866] Updated weights for policy 1, policy_version 92920 (0.0007) -[2023-10-15 18:28:57,504][52833] Updated weights for policy 0, policy_version 92680 (0.0008) -[2023-10-15 18:28:57,871][52833] Updated weights for policy 0, policy_version 92690 (0.0009) -[2023-10-15 18:28:57,913][52866] Updated weights for policy 1, policy_version 92930 (0.0007) -[2023-10-15 18:28:58,252][52833] Updated weights for policy 0, policy_version 92700 (0.0008) -[2023-10-15 18:28:58,281][52866] Updated weights for policy 1, policy_version 92940 (0.0007) -[2023-10-15 18:28:58,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 190087168. Throughput: 0: 1813.1, 1: 1797.9. Samples: 47532824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:28:58,442][51532] Avg episode reward: [(0, '74.500'), (1, '58.390')] -[2023-10-15 18:28:58,640][52866] Updated weights for policy 1, policy_version 92950 (0.0008) -[2023-10-15 18:28:59,008][52866] Updated weights for policy 1, policy_version 92960 (0.0007) -[2023-10-15 18:29:01,923][52833] Updated weights for policy 0, policy_version 92710 (0.0008) -[2023-10-15 18:29:02,295][52833] Updated weights for policy 0, policy_version 92720 (0.0009) -[2023-10-15 18:29:02,676][52833] Updated weights for policy 0, policy_version 92730 (0.0007) -[2023-10-15 18:29:02,909][52866] Updated weights for policy 1, policy_version 92970 (0.0007) -[2023-10-15 18:29:03,272][52866] Updated weights for policy 1, policy_version 92980 (0.0008) -[2023-10-15 18:29:03,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14329.1). Total num frames: 190152704. Throughput: 0: 1794.9, 1: 1781.8. Samples: 47543634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:03,442][51532] Avg episode reward: [(0, '69.450'), (1, '60.310')] -[2023-10-15 18:29:03,646][52866] Updated weights for policy 1, policy_version 92990 (0.0008) -[2023-10-15 18:29:06,424][52833] Updated weights for policy 0, policy_version 92740 (0.0008) -[2023-10-15 18:29:06,786][52833] Updated weights for policy 0, policy_version 92750 (0.0008) -[2023-10-15 18:29:07,152][52833] Updated weights for policy 0, policy_version 92760 (0.0007) -[2023-10-15 18:29:07,475][52866] Updated weights for policy 1, policy_version 93000 (0.0007) -[2023-10-15 18:29:07,849][52866] Updated weights for policy 1, policy_version 93010 (0.0007) -[2023-10-15 18:29:08,210][52866] Updated weights for policy 1, policy_version 93020 (0.0007) -[2023-10-15 18:29:08,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 190251008. Throughput: 0: 1804.3, 1: 1801.6. Samples: 47565018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:08,442][51532] Avg episode reward: [(0, '73.120'), (1, '58.690')] -[2023-10-15 18:29:10,871][52833] Updated weights for policy 0, policy_version 92770 (0.0009) -[2023-10-15 18:29:11,237][52833] Updated weights for policy 0, policy_version 92780 (0.0008) -[2023-10-15 18:29:11,603][52833] Updated weights for policy 0, policy_version 92790 (0.0008) -[2023-10-15 18:29:11,873][52866] Updated weights for policy 1, policy_version 93030 (0.0007) -[2023-10-15 18:29:11,972][52833] Updated weights for policy 0, policy_version 92800 (0.0007) -[2023-10-15 18:29:12,243][52866] Updated weights for policy 1, policy_version 93040 (0.0009) -[2023-10-15 18:29:12,613][52866] Updated weights for policy 1, policy_version 93050 (0.0008) -[2023-10-15 18:29:13,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 190316544. Throughput: 0: 1786.2, 1: 1777.7. Samples: 47585400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:13,442][51532] Avg episode reward: [(0, '76.230'), (1, '60.210')] -[2023-10-15 18:29:15,701][52833] Updated weights for policy 0, policy_version 92810 (0.0007) -[2023-10-15 18:29:16,067][52833] Updated weights for policy 0, policy_version 92820 (0.0009) -[2023-10-15 18:29:16,432][52866] Updated weights for policy 1, policy_version 93060 (0.0007) -[2023-10-15 18:29:16,433][52833] Updated weights for policy 0, policy_version 92830 (0.0008) -[2023-10-15 18:29:16,794][52866] Updated weights for policy 1, policy_version 93070 (0.0008) -[2023-10-15 18:29:17,162][52866] Updated weights for policy 1, policy_version 93080 (0.0007) -[2023-10-15 18:29:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 190382080. Throughput: 0: 1799.0, 1: 1797.3. Samples: 47597416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:18,441][51532] Avg episode reward: [(0, '75.540'), (1, '57.550')] -[2023-10-15 18:29:20,035][52833] Updated weights for policy 0, policy_version 92840 (0.0008) -[2023-10-15 18:29:20,407][52833] Updated weights for policy 0, policy_version 92850 (0.0010) -[2023-10-15 18:29:20,772][52833] Updated weights for policy 0, policy_version 92860 (0.0009) -[2023-10-15 18:29:20,864][52866] Updated weights for policy 1, policy_version 93090 (0.0010) -[2023-10-15 18:29:21,240][52866] Updated weights for policy 1, policy_version 93100 (0.0011) -[2023-10-15 18:29:21,603][52866] Updated weights for policy 1, policy_version 93110 (0.0010) -[2023-10-15 18:29:21,964][52866] Updated weights for policy 1, policy_version 93120 (0.0008) -[2023-10-15 18:29:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 190447616. Throughput: 0: 1794.0, 1: 1784.0. Samples: 47618062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:23,442][51532] Avg episode reward: [(0, '74.400'), (1, '54.120')] -[2023-10-15 18:29:24,524][52833] Updated weights for policy 0, policy_version 92870 (0.0007) -[2023-10-15 18:29:24,892][52833] Updated weights for policy 0, policy_version 92880 (0.0007) -[2023-10-15 18:29:25,269][52833] Updated weights for policy 0, policy_version 92890 (0.0009) -[2023-10-15 18:29:25,710][52866] Updated weights for policy 1, policy_version 93130 (0.0007) -[2023-10-15 18:29:26,073][52866] Updated weights for policy 1, policy_version 93140 (0.0008) -[2023-10-15 18:29:26,451][52866] Updated weights for policy 1, policy_version 93150 (0.0008) -[2023-10-15 18:29:28,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 190513152. Throughput: 0: 1791.6, 1: 1784.6. Samples: 47640420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:28,442][51532] Avg episode reward: [(0, '77.480'), (1, '54.700')] -[2023-10-15 18:29:28,941][52833] Updated weights for policy 0, policy_version 92900 (0.0007) -[2023-10-15 18:29:29,312][52833] Updated weights for policy 0, policy_version 92910 (0.0008) -[2023-10-15 18:29:29,673][52833] Updated weights for policy 0, policy_version 92920 (0.0008) -[2023-10-15 18:29:30,155][52866] Updated weights for policy 1, policy_version 93160 (0.0009) -[2023-10-15 18:29:30,524][52866] Updated weights for policy 1, policy_version 93170 (0.0007) -[2023-10-15 18:29:30,893][52866] Updated weights for policy 1, policy_version 93180 (0.0010) -[2023-10-15 18:29:33,359][52833] Updated weights for policy 0, policy_version 92930 (0.0008) -[2023-10-15 18:29:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 190578688. Throughput: 0: 1797.4, 1: 1793.2. Samples: 47650580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:33,441][51532] Avg episode reward: [(0, '78.560'), (1, '53.900')] -[2023-10-15 18:29:33,732][52833] Updated weights for policy 0, policy_version 92940 (0.0007) -[2023-10-15 18:29:34,094][52833] Updated weights for policy 0, policy_version 92950 (0.0007) -[2023-10-15 18:29:34,461][52833] Updated weights for policy 0, policy_version 92960 (0.0009) -[2023-10-15 18:29:34,570][52866] Updated weights for policy 1, policy_version 93190 (0.0008) -[2023-10-15 18:29:34,938][52866] Updated weights for policy 1, policy_version 93200 (0.0008) -[2023-10-15 18:29:35,304][52866] Updated weights for policy 1, policy_version 93210 (0.0009) -[2023-10-15 18:29:38,206][52833] Updated weights for policy 0, policy_version 92970 (0.0009) -[2023-10-15 18:29:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 190644224. Throughput: 0: 1799.6, 1: 1797.8. Samples: 47673212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:38,442][51532] Avg episode reward: [(0, '75.980'), (1, '54.630')] -[2023-10-15 18:29:38,578][52833] Updated weights for policy 0, policy_version 92980 (0.0007) -[2023-10-15 18:29:38,951][52833] Updated weights for policy 0, policy_version 92990 (0.0009) -[2023-10-15 18:29:39,080][52866] Updated weights for policy 1, policy_version 93220 (0.0009) -[2023-10-15 18:29:39,441][52866] Updated weights for policy 1, policy_version 93230 (0.0009) -[2023-10-15 18:29:39,802][52866] Updated weights for policy 1, policy_version 93240 (0.0010) -[2023-10-15 18:29:42,936][52833] Updated weights for policy 0, policy_version 93000 (0.0010) -[2023-10-15 18:29:43,302][52833] Updated weights for policy 0, policy_version 93010 (0.0007) -[2023-10-15 18:29:43,402][52866] Updated weights for policy 1, policy_version 93250 (0.0009) -[2023-10-15 18:29:43,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 190709760. Throughput: 0: 1806.9, 1: 1808.9. Samples: 47695536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:43,442][51532] Avg episode reward: [(0, '76.680'), (1, '55.160')] -[2023-10-15 18:29:43,666][52833] Updated weights for policy 0, policy_version 93020 (0.0008) -[2023-10-15 18:29:43,772][52866] Updated weights for policy 1, policy_version 93260 (0.0007) -[2023-10-15 18:29:43,815][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000093024_95256576.pth... -[2023-10-15 18:29:43,844][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000091328_93519872.pth -[2023-10-15 18:29:44,144][52866] Updated weights for policy 1, policy_version 93270 (0.0009) -[2023-10-15 18:29:44,509][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000093280_95518720.pth... -[2023-10-15 18:29:44,513][52866] Updated weights for policy 1, policy_version 93280 (0.0009) -[2023-10-15 18:29:44,538][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000091584_93782016.pth -[2023-10-15 18:29:47,372][52833] Updated weights for policy 0, policy_version 93030 (0.0008) -[2023-10-15 18:29:47,735][52833] Updated weights for policy 0, policy_version 93040 (0.0011) -[2023-10-15 18:29:48,103][52833] Updated weights for policy 0, policy_version 93050 (0.0008) -[2023-10-15 18:29:48,416][52866] Updated weights for policy 1, policy_version 93290 (0.0010) -[2023-10-15 18:29:48,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 190808064. Throughput: 0: 1798.9, 1: 1798.5. Samples: 47705516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:48,441][51532] Avg episode reward: [(0, '79.150'), (1, '54.730')] -[2023-10-15 18:29:48,783][52866] Updated weights for policy 1, policy_version 93300 (0.0010) -[2023-10-15 18:29:49,154][52866] Updated weights for policy 1, policy_version 93310 (0.0007) -[2023-10-15 18:29:51,960][52833] Updated weights for policy 0, policy_version 93060 (0.0009) -[2023-10-15 18:29:52,335][52833] Updated weights for policy 0, policy_version 93070 (0.0009) -[2023-10-15 18:29:52,696][52833] Updated weights for policy 0, policy_version 93080 (0.0008) -[2023-10-15 18:29:52,759][52866] Updated weights for policy 1, policy_version 93320 (0.0008) -[2023-10-15 18:29:53,128][52866] Updated weights for policy 1, policy_version 93330 (0.0007) -[2023-10-15 18:29:53,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 190873600. Throughput: 0: 1807.5, 1: 1806.1. Samples: 47727632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:29:53,442][51532] Avg episode reward: [(0, '78.760'), (1, '55.670')] -[2023-10-15 18:29:53,489][52866] Updated weights for policy 1, policy_version 93340 (0.0007) -[2023-10-15 18:29:56,468][52833] Updated weights for policy 0, policy_version 93090 (0.0008) -[2023-10-15 18:29:56,827][52833] Updated weights for policy 0, policy_version 93100 (0.0008) -[2023-10-15 18:29:57,162][52866] Updated weights for policy 1, policy_version 93350 (0.0008) -[2023-10-15 18:29:57,193][52833] Updated weights for policy 0, policy_version 93110 (0.0009) -[2023-10-15 18:29:57,523][52866] Updated weights for policy 1, policy_version 93360 (0.0007) -[2023-10-15 18:29:57,559][52833] Updated weights for policy 0, policy_version 93120 (0.0008) -[2023-10-15 18:29:57,898][52866] Updated weights for policy 1, policy_version 93370 (0.0008) -[2023-10-15 18:29:58,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14440.2). Total num frames: 190971904. Throughput: 0: 1794.6, 1: 1816.0. Samples: 47747878. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:29:58,441][51532] Avg episode reward: [(0, '77.900'), (1, '58.140')] -[2023-10-15 18:30:01,431][52833] Updated weights for policy 0, policy_version 93130 (0.0007) -[2023-10-15 18:30:01,493][52866] Updated weights for policy 1, policy_version 93380 (0.0007) -[2023-10-15 18:30:01,797][52833] Updated weights for policy 0, policy_version 93140 (0.0008) -[2023-10-15 18:30:01,858][52866] Updated weights for policy 1, policy_version 93390 (0.0007) -[2023-10-15 18:30:02,175][52833] Updated weights for policy 0, policy_version 93150 (0.0008) -[2023-10-15 18:30:02,221][52866] Updated weights for policy 1, policy_version 93400 (0.0008) -[2023-10-15 18:30:03,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 191037440. Throughput: 0: 1808.8, 1: 1819.8. Samples: 47760700. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:03,442][51532] Avg episode reward: [(0, '78.260'), (1, '63.650')] -[2023-10-15 18:30:05,807][52866] Updated weights for policy 1, policy_version 93410 (0.0008) -[2023-10-15 18:30:05,826][52833] Updated weights for policy 0, policy_version 93160 (0.0009) -[2023-10-15 18:30:06,176][52866] Updated weights for policy 1, policy_version 93420 (0.0007) -[2023-10-15 18:30:06,199][52833] Updated weights for policy 0, policy_version 93170 (0.0007) -[2023-10-15 18:30:06,545][52866] Updated weights for policy 1, policy_version 93430 (0.0008) -[2023-10-15 18:30:06,564][52833] Updated weights for policy 0, policy_version 93180 (0.0007) -[2023-10-15 18:30:06,911][52866] Updated weights for policy 1, policy_version 93440 (0.0007) -[2023-10-15 18:30:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 191102976. Throughput: 0: 1787.2, 1: 1818.7. Samples: 47780328. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:08,442][51532] Avg episode reward: [(0, '75.420'), (1, '61.500')] -[2023-10-15 18:30:10,314][52833] Updated weights for policy 0, policy_version 93190 (0.0010) -[2023-10-15 18:30:10,691][52833] Updated weights for policy 0, policy_version 93200 (0.0010) -[2023-10-15 18:30:10,767][52866] Updated weights for policy 1, policy_version 93450 (0.0007) -[2023-10-15 18:30:11,065][52833] Updated weights for policy 0, policy_version 93210 (0.0010) -[2023-10-15 18:30:11,136][52866] Updated weights for policy 1, policy_version 93460 (0.0008) -[2023-10-15 18:30:11,504][52866] Updated weights for policy 1, policy_version 93470 (0.0008) -[2023-10-15 18:30:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191168512. Throughput: 0: 1787.4, 1: 1815.2. Samples: 47802538. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:13,442][51532] Avg episode reward: [(0, '72.950'), (1, '61.600')] -[2023-10-15 18:30:14,976][52833] Updated weights for policy 0, policy_version 93220 (0.0007) -[2023-10-15 18:30:15,202][52866] Updated weights for policy 1, policy_version 93480 (0.0008) -[2023-10-15 18:30:15,343][52833] Updated weights for policy 0, policy_version 93230 (0.0009) -[2023-10-15 18:30:15,570][52866] Updated weights for policy 1, policy_version 93490 (0.0008) -[2023-10-15 18:30:15,716][52833] Updated weights for policy 0, policy_version 93240 (0.0009) -[2023-10-15 18:30:15,932][52866] Updated weights for policy 1, policy_version 93500 (0.0008) -[2023-10-15 18:30:18,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 191234048. Throughput: 0: 1790.4, 1: 1811.6. Samples: 47812672. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:18,442][51532] Avg episode reward: [(0, '76.800'), (1, '67.210')] -[2023-10-15 18:30:19,460][52833] Updated weights for policy 0, policy_version 93250 (0.0007) -[2023-10-15 18:30:19,795][52866] Updated weights for policy 1, policy_version 93510 (0.0009) -[2023-10-15 18:30:19,828][52833] Updated weights for policy 0, policy_version 93260 (0.0007) -[2023-10-15 18:30:20,169][52866] Updated weights for policy 1, policy_version 93520 (0.0008) -[2023-10-15 18:30:20,201][52833] Updated weights for policy 0, policy_version 93270 (0.0007) -[2023-10-15 18:30:20,539][52866] Updated weights for policy 1, policy_version 93530 (0.0009) -[2023-10-15 18:30:20,557][52833] Updated weights for policy 0, policy_version 93280 (0.0009) -[2023-10-15 18:30:23,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 191299584. Throughput: 0: 1778.4, 1: 1802.7. Samples: 47834360. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:23,442][51532] Avg episode reward: [(0, '75.570'), (1, '68.410')] -[2023-10-15 18:30:24,194][52833] Updated weights for policy 0, policy_version 93290 (0.0009) -[2023-10-15 18:30:24,308][52866] Updated weights for policy 1, policy_version 93540 (0.0008) -[2023-10-15 18:30:24,555][52833] Updated weights for policy 0, policy_version 93300 (0.0007) -[2023-10-15 18:30:24,683][52866] Updated weights for policy 1, policy_version 93550 (0.0008) -[2023-10-15 18:30:24,920][52833] Updated weights for policy 0, policy_version 93310 (0.0009) -[2023-10-15 18:30:25,043][52866] Updated weights for policy 1, policy_version 93560 (0.0009) -[2023-10-15 18:30:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 191365120. Throughput: 0: 1793.5, 1: 1798.5. Samples: 47857178. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:28,442][51532] Avg episode reward: [(0, '76.150'), (1, '72.860')] -[2023-10-15 18:30:28,750][52833] Updated weights for policy 0, policy_version 93320 (0.0009) -[2023-10-15 18:30:28,882][52866] Updated weights for policy 1, policy_version 93570 (0.0008) -[2023-10-15 18:30:29,130][52833] Updated weights for policy 0, policy_version 93330 (0.0008) -[2023-10-15 18:30:29,245][52866] Updated weights for policy 1, policy_version 93580 (0.0007) -[2023-10-15 18:30:29,491][52833] Updated weights for policy 0, policy_version 93340 (0.0007) -[2023-10-15 18:30:29,608][52866] Updated weights for policy 1, policy_version 93590 (0.0009) -[2023-10-15 18:30:29,975][52866] Updated weights for policy 1, policy_version 93600 (0.0010) -[2023-10-15 18:30:33,158][52833] Updated weights for policy 0, policy_version 93350 (0.0007) -[2023-10-15 18:30:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 191430656. Throughput: 0: 1781.6, 1: 1802.0. Samples: 47866782. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:33,442][51532] Avg episode reward: [(0, '73.760'), (1, '72.070')] -[2023-10-15 18:30:33,521][52833] Updated weights for policy 0, policy_version 93360 (0.0007) -[2023-10-15 18:30:33,802][52866] Updated weights for policy 1, policy_version 93610 (0.0013) -[2023-10-15 18:30:33,884][52833] Updated weights for policy 0, policy_version 93370 (0.0009) -[2023-10-15 18:30:34,162][52866] Updated weights for policy 1, policy_version 93620 (0.0009) -[2023-10-15 18:30:34,534][52866] Updated weights for policy 1, policy_version 93630 (0.0010) -[2023-10-15 18:30:37,709][52833] Updated weights for policy 0, policy_version 93380 (0.0008) -[2023-10-15 18:30:38,078][52833] Updated weights for policy 0, policy_version 93390 (0.0008) -[2023-10-15 18:30:38,208][52866] Updated weights for policy 1, policy_version 93640 (0.0008) -[2023-10-15 18:30:38,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 191496192. Throughput: 0: 1788.6, 1: 1798.8. Samples: 47889064. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:38,441][51532] Avg episode reward: [(0, '71.540'), (1, '73.720')] -[2023-10-15 18:30:38,449][52833] Updated weights for policy 0, policy_version 93400 (0.0009) -[2023-10-15 18:30:38,579][52866] Updated weights for policy 1, policy_version 93650 (0.0008) -[2023-10-15 18:30:38,943][52866] Updated weights for policy 1, policy_version 93660 (0.0008) -[2023-10-15 18:30:42,260][52833] Updated weights for policy 0, policy_version 93410 (0.0007) -[2023-10-15 18:30:42,633][52833] Updated weights for policy 0, policy_version 93420 (0.0007) -[2023-10-15 18:30:42,730][52866] Updated weights for policy 1, policy_version 93670 (0.0009) -[2023-10-15 18:30:43,000][52833] Updated weights for policy 0, policy_version 93430 (0.0008) -[2023-10-15 18:30:43,100][52866] Updated weights for policy 1, policy_version 93680 (0.0007) -[2023-10-15 18:30:43,366][52833] Updated weights for policy 0, policy_version 93440 (0.0009) -[2023-10-15 18:30:43,441][51532] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 191594496. Throughput: 0: 1800.5, 1: 1805.7. Samples: 47910158. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:43,442][51532] Avg episode reward: [(0, '71.690'), (1, '72.870')] -[2023-10-15 18:30:43,463][52866] Updated weights for policy 1, policy_version 93690 (0.0008) -[2023-10-15 18:30:47,142][52833] Updated weights for policy 0, policy_version 93450 (0.0009) -[2023-10-15 18:30:47,221][52866] Updated weights for policy 1, policy_version 93700 (0.0009) -[2023-10-15 18:30:47,516][52833] Updated weights for policy 0, policy_version 93460 (0.0008) -[2023-10-15 18:30:47,585][52866] Updated weights for policy 1, policy_version 93710 (0.0007) -[2023-10-15 18:30:47,886][52833] Updated weights for policy 0, policy_version 93470 (0.0009) -[2023-10-15 18:30:47,954][52866] Updated weights for policy 1, policy_version 93720 (0.0008) -[2023-10-15 18:30:48,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 191692800. Throughput: 0: 1782.6, 1: 1782.8. Samples: 47921140. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:48,441][51532] Avg episode reward: [(0, '73.230'), (1, '76.090')] -[2023-10-15 18:30:51,623][52833] Updated weights for policy 0, policy_version 93480 (0.0008) -[2023-10-15 18:30:51,650][52866] Updated weights for policy 1, policy_version 93730 (0.0009) -[2023-10-15 18:30:51,987][52833] Updated weights for policy 0, policy_version 93490 (0.0007) -[2023-10-15 18:30:52,012][52866] Updated weights for policy 1, policy_version 93740 (0.0008) -[2023-10-15 18:30:52,359][52833] Updated weights for policy 0, policy_version 93500 (0.0007) -[2023-10-15 18:30:52,383][52866] Updated weights for policy 1, policy_version 93750 (0.0007) -[2023-10-15 18:30:52,750][52866] Updated weights for policy 1, policy_version 93760 (0.0007) -[2023-10-15 18:30:53,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 191758336. Throughput: 0: 1798.3, 1: 1802.1. Samples: 47942346. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) -[2023-10-15 18:30:53,442][51532] Avg episode reward: [(0, '72.170'), (1, '77.530')] -[2023-10-15 18:30:53,442][52518] Saving new best policy, reward=77.530! -[2023-10-15 18:30:55,944][52833] Updated weights for policy 0, policy_version 93510 (0.0008) -[2023-10-15 18:30:56,308][52833] Updated weights for policy 0, policy_version 93520 (0.0010) -[2023-10-15 18:30:56,430][52866] Updated weights for policy 1, policy_version 93770 (0.0007) -[2023-10-15 18:30:56,677][52833] Updated weights for policy 0, policy_version 93530 (0.0008) -[2023-10-15 18:30:56,800][52866] Updated weights for policy 1, policy_version 93780 (0.0007) -[2023-10-15 18:30:57,164][52866] Updated weights for policy 1, policy_version 93790 (0.0007) -[2023-10-15 18:30:58,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191823872. Throughput: 0: 1786.2, 1: 1787.3. Samples: 47963346. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:30:58,441][51532] Avg episode reward: [(0, '73.890'), (1, '78.450')] -[2023-10-15 18:30:58,451][52518] Saving new best policy, reward=78.450! -[2023-10-15 18:31:00,516][52833] Updated weights for policy 0, policy_version 93540 (0.0008) -[2023-10-15 18:31:00,829][52866] Updated weights for policy 1, policy_version 93800 (0.0007) -[2023-10-15 18:31:00,886][52833] Updated weights for policy 0, policy_version 93550 (0.0010) -[2023-10-15 18:31:01,186][52866] Updated weights for policy 1, policy_version 93810 (0.0008) -[2023-10-15 18:31:01,260][52833] Updated weights for policy 0, policy_version 93560 (0.0010) -[2023-10-15 18:31:01,546][52866] Updated weights for policy 1, policy_version 93820 (0.0008) -[2023-10-15 18:31:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191889408. Throughput: 0: 1796.9, 1: 1806.5. Samples: 47974824. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:03,441][51532] Avg episode reward: [(0, '73.850'), (1, '78.610')] -[2023-10-15 18:31:03,442][52518] Saving new best policy, reward=78.610! -[2023-10-15 18:31:04,883][52833] Updated weights for policy 0, policy_version 93570 (0.0009) -[2023-10-15 18:31:05,099][52866] Updated weights for policy 1, policy_version 93830 (0.0008) -[2023-10-15 18:31:05,258][52833] Updated weights for policy 0, policy_version 93580 (0.0009) -[2023-10-15 18:31:05,455][52866] Updated weights for policy 1, policy_version 93840 (0.0008) -[2023-10-15 18:31:05,626][52833] Updated weights for policy 0, policy_version 93590 (0.0007) -[2023-10-15 18:31:05,826][52866] Updated weights for policy 1, policy_version 93850 (0.0008) -[2023-10-15 18:31:05,993][52833] Updated weights for policy 0, policy_version 93600 (0.0007) -[2023-10-15 18:31:08,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191954944. Throughput: 0: 1791.8, 1: 1801.3. Samples: 47996050. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:08,442][51532] Avg episode reward: [(0, '76.200'), (1, '73.000')] -[2023-10-15 18:31:09,602][52866] Updated weights for policy 1, policy_version 93860 (0.0010) -[2023-10-15 18:31:09,845][52833] Updated weights for policy 0, policy_version 93610 (0.0009) -[2023-10-15 18:31:09,967][52866] Updated weights for policy 1, policy_version 93870 (0.0008) -[2023-10-15 18:31:10,214][52833] Updated weights for policy 0, policy_version 93620 (0.0009) -[2023-10-15 18:31:10,326][52866] Updated weights for policy 1, policy_version 93880 (0.0009) -[2023-10-15 18:31:10,575][52833] Updated weights for policy 0, policy_version 93630 (0.0007) -[2023-10-15 18:31:13,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 192020480. Throughput: 0: 1781.7, 1: 1806.6. Samples: 48018654. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:13,441][51532] Avg episode reward: [(0, '77.760'), (1, '75.630')] -[2023-10-15 18:31:14,151][52866] Updated weights for policy 1, policy_version 93890 (0.0007) -[2023-10-15 18:31:14,515][52866] Updated weights for policy 1, policy_version 93900 (0.0007) -[2023-10-15 18:31:14,539][52833] Updated weights for policy 0, policy_version 93640 (0.0008) -[2023-10-15 18:31:14,874][52866] Updated weights for policy 1, policy_version 93910 (0.0008) -[2023-10-15 18:31:14,911][52833] Updated weights for policy 0, policy_version 93650 (0.0007) -[2023-10-15 18:31:15,237][52866] Updated weights for policy 1, policy_version 93920 (0.0009) -[2023-10-15 18:31:15,276][52833] Updated weights for policy 0, policy_version 93660 (0.0008) -[2023-10-15 18:31:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.0). Total num frames: 192086016. Throughput: 0: 1782.6, 1: 1805.4. Samples: 48028242. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:18,442][51532] Avg episode reward: [(0, '80.460'), (1, '72.930')] -[2023-10-15 18:31:18,912][52833] Updated weights for policy 0, policy_version 93670 (0.0008) -[2023-10-15 18:31:19,094][52866] Updated weights for policy 1, policy_version 93930 (0.0010) -[2023-10-15 18:31:19,285][52833] Updated weights for policy 0, policy_version 93680 (0.0009) -[2023-10-15 18:31:19,454][52866] Updated weights for policy 1, policy_version 93940 (0.0007) -[2023-10-15 18:31:19,646][52833] Updated weights for policy 0, policy_version 93690 (0.0009) -[2023-10-15 18:31:19,817][52866] Updated weights for policy 1, policy_version 93950 (0.0009) -[2023-10-15 18:31:23,388][52833] Updated weights for policy 0, policy_version 93700 (0.0008) -[2023-10-15 18:31:23,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 192151552. Throughput: 0: 1785.0, 1: 1804.2. Samples: 48050578. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:23,442][51532] Avg episode reward: [(0, '77.070'), (1, '72.420')] -[2023-10-15 18:31:23,607][52866] Updated weights for policy 1, policy_version 93960 (0.0008) -[2023-10-15 18:31:23,769][52833] Updated weights for policy 0, policy_version 93710 (0.0008) -[2023-10-15 18:31:23,972][52866] Updated weights for policy 1, policy_version 93970 (0.0008) -[2023-10-15 18:31:24,136][52833] Updated weights for policy 0, policy_version 93720 (0.0009) -[2023-10-15 18:31:24,335][52866] Updated weights for policy 1, policy_version 93980 (0.0009) -[2023-10-15 18:31:28,055][52833] Updated weights for policy 0, policy_version 93730 (0.0008) -[2023-10-15 18:31:28,093][52866] Updated weights for policy 1, policy_version 93990 (0.0007) -[2023-10-15 18:31:28,420][52833] Updated weights for policy 0, policy_version 93740 (0.0009) -[2023-10-15 18:31:28,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 192217088. Throughput: 0: 1800.0, 1: 1815.2. Samples: 48072840. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:28,441][51532] Avg episode reward: [(0, '77.020'), (1, '72.280')] -[2023-10-15 18:31:28,462][52866] Updated weights for policy 1, policy_version 94000 (0.0007) -[2023-10-15 18:31:28,794][52833] Updated weights for policy 0, policy_version 93750 (0.0010) -[2023-10-15 18:31:28,831][52866] Updated weights for policy 1, policy_version 94010 (0.0007) -[2023-10-15 18:31:29,174][52833] Updated weights for policy 0, policy_version 93760 (0.0009) -[2023-10-15 18:31:32,548][52866] Updated weights for policy 1, policy_version 94020 (0.0008) -[2023-10-15 18:31:32,785][52833] Updated weights for policy 0, policy_version 93770 (0.0007) -[2023-10-15 18:31:32,906][52866] Updated weights for policy 1, policy_version 94030 (0.0008) -[2023-10-15 18:31:33,159][52833] Updated weights for policy 0, policy_version 93780 (0.0008) -[2023-10-15 18:31:33,276][52866] Updated weights for policy 1, policy_version 94040 (0.0007) -[2023-10-15 18:31:33,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 192282624. Throughput: 0: 1783.7, 1: 1804.3. Samples: 48082602. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:33,441][51532] Avg episode reward: [(0, '77.380'), (1, '73.470')] -[2023-10-15 18:31:33,530][52833] Updated weights for policy 0, policy_version 93790 (0.0008) -[2023-10-15 18:31:36,964][52866] Updated weights for policy 1, policy_version 94050 (0.0007) -[2023-10-15 18:31:37,329][52866] Updated weights for policy 1, policy_version 94060 (0.0009) -[2023-10-15 18:31:37,405][52833] Updated weights for policy 0, policy_version 93800 (0.0008) -[2023-10-15 18:31:37,703][52866] Updated weights for policy 1, policy_version 94070 (0.0007) -[2023-10-15 18:31:37,775][52833] Updated weights for policy 0, policy_version 93810 (0.0009) -[2023-10-15 18:31:38,059][52866] Updated weights for policy 1, policy_version 94080 (0.0010) -[2023-10-15 18:31:38,140][52833] Updated weights for policy 0, policy_version 93820 (0.0008) -[2023-10-15 18:31:38,441][51532] Fps is (10 sec: 19660.2, 60 sec: 15291.6, 300 sec: 14440.1). Total num frames: 192413696. Throughput: 0: 1797.0, 1: 1817.2. Samples: 48104988. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:38,442][51532] Avg episode reward: [(0, '73.350'), (1, '75.250')] -[2023-10-15 18:31:41,742][52866] Updated weights for policy 1, policy_version 94090 (0.0009) -[2023-10-15 18:31:41,808][52833] Updated weights for policy 0, policy_version 93830 (0.0008) -[2023-10-15 18:31:42,112][52866] Updated weights for policy 1, policy_version 94100 (0.0009) -[2023-10-15 18:31:42,174][52833] Updated weights for policy 0, policy_version 93840 (0.0009) -[2023-10-15 18:31:42,480][52866] Updated weights for policy 1, policy_version 94110 (0.0008) -[2023-10-15 18:31:42,549][52833] Updated weights for policy 0, policy_version 93850 (0.0007) -[2023-10-15 18:31:43,441][51532] Fps is (10 sec: 19660.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 192479232. Throughput: 0: 1779.9, 1: 1801.7. Samples: 48124518. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:43,442][51532] Avg episode reward: [(0, '74.480'), (1, '72.660')] -[2023-10-15 18:31:43,453][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000094112_96370688.pth... -[2023-10-15 18:31:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000093856_96108544.pth... -[2023-10-15 18:31:43,484][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000092416_94633984.pth -[2023-10-15 18:31:43,489][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000092160_94371840.pth -[2023-10-15 18:31:46,240][52866] Updated weights for policy 1, policy_version 94120 (0.0009) -[2023-10-15 18:31:46,375][52833] Updated weights for policy 0, policy_version 93860 (0.0007) -[2023-10-15 18:31:46,600][52866] Updated weights for policy 1, policy_version 94130 (0.0009) -[2023-10-15 18:31:46,740][52833] Updated weights for policy 0, policy_version 93870 (0.0009) -[2023-10-15 18:31:46,971][52866] Updated weights for policy 1, policy_version 94140 (0.0008) -[2023-10-15 18:31:47,112][52833] Updated weights for policy 0, policy_version 93880 (0.0008) -[2023-10-15 18:31:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192544768. Throughput: 0: 1799.7, 1: 1812.1. Samples: 48137356. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:48,441][51532] Avg episode reward: [(0, '74.100'), (1, '72.990')] -[2023-10-15 18:31:50,836][52866] Updated weights for policy 1, policy_version 94150 (0.0008) -[2023-10-15 18:31:51,025][52833] Updated weights for policy 0, policy_version 93890 (0.0007) -[2023-10-15 18:31:51,202][52866] Updated weights for policy 1, policy_version 94160 (0.0009) -[2023-10-15 18:31:51,394][52833] Updated weights for policy 0, policy_version 93900 (0.0009) -[2023-10-15 18:31:51,581][52866] Updated weights for policy 1, policy_version 94170 (0.0010) -[2023-10-15 18:31:51,754][52833] Updated weights for policy 0, policy_version 93910 (0.0007) -[2023-10-15 18:31:52,127][52833] Updated weights for policy 0, policy_version 93920 (0.0009) -[2023-10-15 18:31:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 192610304. Throughput: 0: 1789.6, 1: 1789.5. Samples: 48157108. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-15 18:31:53,443][51532] Avg episode reward: [(0, '73.470'), (1, '72.810')] -[2023-10-15 18:31:55,299][52866] Updated weights for policy 1, policy_version 94180 (0.0008) -[2023-10-15 18:31:55,675][52866] Updated weights for policy 1, policy_version 94190 (0.0008) -[2023-10-15 18:31:55,766][52833] Updated weights for policy 0, policy_version 93930 (0.0008) -[2023-10-15 18:31:56,038][52866] Updated weights for policy 1, policy_version 94200 (0.0007) -[2023-10-15 18:31:56,138][52833] Updated weights for policy 0, policy_version 93940 (0.0008) -[2023-10-15 18:31:56,507][52833] Updated weights for policy 0, policy_version 93950 (0.0009) -[2023-10-15 18:31:58,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 192675840. Throughput: 0: 1788.5, 1: 1786.5. Samples: 48179532. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:31:58,442][51532] Avg episode reward: [(0, '73.410'), (1, '70.960')] -[2023-10-15 18:31:59,779][52866] Updated weights for policy 1, policy_version 94210 (0.0008) -[2023-10-15 18:32:00,145][52866] Updated weights for policy 1, policy_version 94220 (0.0008) -[2023-10-15 18:32:00,168][52833] Updated weights for policy 0, policy_version 93960 (0.0008) -[2023-10-15 18:32:00,506][52866] Updated weights for policy 1, policy_version 94230 (0.0008) -[2023-10-15 18:32:00,542][52833] Updated weights for policy 0, policy_version 93970 (0.0009) -[2023-10-15 18:32:00,868][52866] Updated weights for policy 1, policy_version 94240 (0.0010) -[2023-10-15 18:32:00,909][52833] Updated weights for policy 0, policy_version 93980 (0.0009) -[2023-10-15 18:32:03,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192741376. Throughput: 0: 1797.1, 1: 1785.6. Samples: 48189464. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:03,441][51532] Avg episode reward: [(0, '75.460'), (1, '70.190')] -[2023-10-15 18:32:04,710][52866] Updated weights for policy 1, policy_version 94250 (0.0008) -[2023-10-15 18:32:04,749][52833] Updated weights for policy 0, policy_version 93990 (0.0009) -[2023-10-15 18:32:05,078][52866] Updated weights for policy 1, policy_version 94260 (0.0007) -[2023-10-15 18:32:05,113][52833] Updated weights for policy 0, policy_version 94000 (0.0007) -[2023-10-15 18:32:05,442][52866] Updated weights for policy 1, policy_version 94270 (0.0007) -[2023-10-15 18:32:05,482][52833] Updated weights for policy 0, policy_version 94010 (0.0007) -[2023-10-15 18:32:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192806912. Throughput: 0: 1790.0, 1: 1784.7. Samples: 48211438. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:08,441][51532] Avg episode reward: [(0, '73.070'), (1, '71.900')] -[2023-10-15 18:32:09,148][52833] Updated weights for policy 0, policy_version 94020 (0.0009) -[2023-10-15 18:32:09,358][52866] Updated weights for policy 1, policy_version 94280 (0.0008) -[2023-10-15 18:32:09,517][52833] Updated weights for policy 0, policy_version 94030 (0.0008) -[2023-10-15 18:32:09,722][52866] Updated weights for policy 1, policy_version 94290 (0.0007) -[2023-10-15 18:32:09,887][52833] Updated weights for policy 0, policy_version 94040 (0.0009) -[2023-10-15 18:32:10,083][52866] Updated weights for policy 1, policy_version 94300 (0.0007) -[2023-10-15 18:32:13,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 192872448. Throughput: 0: 1791.1, 1: 1786.4. Samples: 48233830. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:13,442][51532] Avg episode reward: [(0, '73.270'), (1, '70.270')] -[2023-10-15 18:32:13,634][52833] Updated weights for policy 0, policy_version 94050 (0.0008) -[2023-10-15 18:32:13,969][52866] Updated weights for policy 1, policy_version 94310 (0.0008) -[2023-10-15 18:32:14,001][52833] Updated weights for policy 0, policy_version 94060 (0.0008) -[2023-10-15 18:32:14,332][52866] Updated weights for policy 1, policy_version 94320 (0.0009) -[2023-10-15 18:32:14,376][52833] Updated weights for policy 0, policy_version 94070 (0.0007) -[2023-10-15 18:32:14,692][52866] Updated weights for policy 1, policy_version 94330 (0.0008) -[2023-10-15 18:32:14,739][52833] Updated weights for policy 0, policy_version 94080 (0.0008) -[2023-10-15 18:32:18,399][52833] Updated weights for policy 0, policy_version 94090 (0.0008) -[2023-10-15 18:32:18,417][52866] Updated weights for policy 1, policy_version 94340 (0.0008) -[2023-10-15 18:32:18,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 192937984. Throughput: 0: 1792.0, 1: 1787.5. Samples: 48243680. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:18,442][51532] Avg episode reward: [(0, '71.470'), (1, '69.610')] -[2023-10-15 18:32:18,754][52833] Updated weights for policy 0, policy_version 94100 (0.0008) -[2023-10-15 18:32:18,793][52866] Updated weights for policy 1, policy_version 94350 (0.0009) -[2023-10-15 18:32:19,125][52833] Updated weights for policy 0, policy_version 94110 (0.0008) -[2023-10-15 18:32:19,152][52866] Updated weights for policy 1, policy_version 94360 (0.0008) -[2023-10-15 18:32:22,855][52866] Updated weights for policy 1, policy_version 94370 (0.0008) -[2023-10-15 18:32:23,005][52833] Updated weights for policy 0, policy_version 94120 (0.0007) -[2023-10-15 18:32:23,229][52866] Updated weights for policy 1, policy_version 94380 (0.0008) -[2023-10-15 18:32:23,380][52833] Updated weights for policy 0, policy_version 94130 (0.0007) -[2023-10-15 18:32:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 193003520. Throughput: 0: 1794.6, 1: 1786.3. Samples: 48266128. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:23,442][51532] Avg episode reward: [(0, '70.420'), (1, '69.180')] -[2023-10-15 18:32:23,598][52866] Updated weights for policy 1, policy_version 94390 (0.0007) -[2023-10-15 18:32:23,754][52833] Updated weights for policy 0, policy_version 94140 (0.0007) -[2023-10-15 18:32:23,962][52866] Updated weights for policy 1, policy_version 94400 (0.0008) -[2023-10-15 18:32:27,532][52833] Updated weights for policy 0, policy_version 94150 (0.0007) -[2023-10-15 18:32:27,760][52866] Updated weights for policy 1, policy_version 94410 (0.0008) -[2023-10-15 18:32:27,894][52833] Updated weights for policy 0, policy_version 94160 (0.0008) -[2023-10-15 18:32:28,124][52866] Updated weights for policy 1, policy_version 94420 (0.0009) -[2023-10-15 18:32:28,251][52833] Updated weights for policy 0, policy_version 94170 (0.0007) -[2023-10-15 18:32:28,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 193069056. Throughput: 0: 1811.2, 1: 1801.5. Samples: 48287090. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:28,442][51532] Avg episode reward: [(0, '69.390'), (1, '66.820')] -[2023-10-15 18:32:28,497][52866] Updated weights for policy 1, policy_version 94430 (0.0008) -[2023-10-15 18:32:32,126][52833] Updated weights for policy 0, policy_version 94180 (0.0009) -[2023-10-15 18:32:32,134][52866] Updated weights for policy 1, policy_version 94440 (0.0007) -[2023-10-15 18:32:32,497][52866] Updated weights for policy 1, policy_version 94450 (0.0009) -[2023-10-15 18:32:32,502][52833] Updated weights for policy 0, policy_version 94190 (0.0008) -[2023-10-15 18:32:32,858][52866] Updated weights for policy 1, policy_version 94460 (0.0007) -[2023-10-15 18:32:32,859][52833] Updated weights for policy 0, policy_version 94200 (0.0009) -[2023-10-15 18:32:33,441][51532] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 193200128. Throughput: 0: 1787.1, 1: 1782.9. Samples: 48298004. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:33,442][51532] Avg episode reward: [(0, '71.800'), (1, '64.430')] -[2023-10-15 18:32:36,572][52833] Updated weights for policy 0, policy_version 94210 (0.0008) -[2023-10-15 18:32:36,594][52866] Updated weights for policy 1, policy_version 94470 (0.0009) -[2023-10-15 18:32:36,943][52833] Updated weights for policy 0, policy_version 94220 (0.0007) -[2023-10-15 18:32:36,951][52866] Updated weights for policy 1, policy_version 94480 (0.0011) -[2023-10-15 18:32:37,312][52833] Updated weights for policy 0, policy_version 94230 (0.0009) -[2023-10-15 18:32:37,320][52866] Updated weights for policy 1, policy_version 94490 (0.0007) -[2023-10-15 18:32:37,687][52833] Updated weights for policy 0, policy_version 94240 (0.0009) -[2023-10-15 18:32:38,441][51532] Fps is (10 sec: 19660.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193265664. Throughput: 0: 1801.7, 1: 1801.4. Samples: 48319248. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:38,442][51532] Avg episode reward: [(0, '74.470'), (1, '64.510')] -[2023-10-15 18:32:41,016][52866] Updated weights for policy 1, policy_version 94500 (0.0008) -[2023-10-15 18:32:41,378][52866] Updated weights for policy 1, policy_version 94510 (0.0009) -[2023-10-15 18:32:41,479][52833] Updated weights for policy 0, policy_version 94250 (0.0007) -[2023-10-15 18:32:41,740][52866] Updated weights for policy 1, policy_version 94520 (0.0008) -[2023-10-15 18:32:41,853][52833] Updated weights for policy 0, policy_version 94260 (0.0008) -[2023-10-15 18:32:42,217][52833] Updated weights for policy 0, policy_version 94270 (0.0007) -[2023-10-15 18:32:43,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193331200. Throughput: 0: 1788.2, 1: 1780.4. Samples: 48340118. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:43,441][51532] Avg episode reward: [(0, '78.170'), (1, '66.300')] -[2023-10-15 18:32:45,646][52866] Updated weights for policy 1, policy_version 94530 (0.0008) -[2023-10-15 18:32:45,763][52833] Updated weights for policy 0, policy_version 94280 (0.0008) -[2023-10-15 18:32:46,009][52866] Updated weights for policy 1, policy_version 94540 (0.0007) -[2023-10-15 18:32:46,142][52833] Updated weights for policy 0, policy_version 94290 (0.0008) -[2023-10-15 18:32:46,369][52866] Updated weights for policy 1, policy_version 94550 (0.0009) -[2023-10-15 18:32:46,500][52833] Updated weights for policy 0, policy_version 94300 (0.0008) -[2023-10-15 18:32:46,735][52866] Updated weights for policy 1, policy_version 94560 (0.0009) -[2023-10-15 18:32:48,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193396736. Throughput: 0: 1804.4, 1: 1802.4. Samples: 48351766. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:48,441][51532] Avg episode reward: [(0, '78.750'), (1, '61.030')] -[2023-10-15 18:32:50,283][52833] Updated weights for policy 0, policy_version 94310 (0.0008) -[2023-10-15 18:32:50,483][52866] Updated weights for policy 1, policy_version 94570 (0.0008) -[2023-10-15 18:32:50,653][52833] Updated weights for policy 0, policy_version 94320 (0.0008) -[2023-10-15 18:32:50,850][52866] Updated weights for policy 1, policy_version 94580 (0.0007) -[2023-10-15 18:32:51,013][52833] Updated weights for policy 0, policy_version 94330 (0.0008) -[2023-10-15 18:32:51,221][52866] Updated weights for policy 1, policy_version 94590 (0.0007) -[2023-10-15 18:32:53,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 193462272. Throughput: 0: 1786.0, 1: 1781.5. Samples: 48371978. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) -[2023-10-15 18:32:53,441][51532] Avg episode reward: [(0, '78.250'), (1, '60.760')] -[2023-10-15 18:32:54,674][52833] Updated weights for policy 0, policy_version 94340 (0.0009) -[2023-10-15 18:32:55,034][52833] Updated weights for policy 0, policy_version 94350 (0.0010) -[2023-10-15 18:32:55,261][52866] Updated weights for policy 1, policy_version 94600 (0.0009) -[2023-10-15 18:32:55,418][52833] Updated weights for policy 0, policy_version 94360 (0.0008) -[2023-10-15 18:32:55,634][52866] Updated weights for policy 1, policy_version 94610 (0.0009) -[2023-10-15 18:32:56,000][52866] Updated weights for policy 1, policy_version 94620 (0.0009) -[2023-10-15 18:32:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 193527808. Throughput: 0: 1789.3, 1: 1776.3. Samples: 48394286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:32:58,442][51532] Avg episode reward: [(0, '74.470'), (1, '60.640')] -[2023-10-15 18:32:59,209][52833] Updated weights for policy 0, policy_version 94370 (0.0007) -[2023-10-15 18:32:59,577][52833] Updated weights for policy 0, policy_version 94380 (0.0008) -[2023-10-15 18:32:59,900][52866] Updated weights for policy 1, policy_version 94630 (0.0008) -[2023-10-15 18:32:59,949][52833] Updated weights for policy 0, policy_version 94390 (0.0008) -[2023-10-15 18:33:00,261][52866] Updated weights for policy 1, policy_version 94640 (0.0008) -[2023-10-15 18:33:00,315][52833] Updated weights for policy 0, policy_version 94400 (0.0008) -[2023-10-15 18:33:00,622][52866] Updated weights for policy 1, policy_version 94650 (0.0009) -[2023-10-15 18:33:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 193593344. Throughput: 0: 1794.5, 1: 1773.2. Samples: 48404226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:03,442][51532] Avg episode reward: [(0, '71.770'), (1, '62.930')] -[2023-10-15 18:33:04,171][52833] Updated weights for policy 0, policy_version 94410 (0.0008) -[2023-10-15 18:33:04,344][52866] Updated weights for policy 1, policy_version 94660 (0.0009) -[2023-10-15 18:33:04,549][52833] Updated weights for policy 0, policy_version 94420 (0.0007) -[2023-10-15 18:33:04,701][52866] Updated weights for policy 1, policy_version 94670 (0.0008) -[2023-10-15 18:33:04,922][52833] Updated weights for policy 0, policy_version 94430 (0.0009) -[2023-10-15 18:33:05,072][52866] Updated weights for policy 1, policy_version 94680 (0.0009) -[2023-10-15 18:33:08,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 193658880. Throughput: 0: 1794.8, 1: 1771.5. Samples: 48426610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:08,442][51532] Avg episode reward: [(0, '74.580'), (1, '63.490')] -[2023-10-15 18:33:08,650][52833] Updated weights for policy 0, policy_version 94440 (0.0007) -[2023-10-15 18:33:08,886][52866] Updated weights for policy 1, policy_version 94690 (0.0009) -[2023-10-15 18:33:09,024][52833] Updated weights for policy 0, policy_version 94450 (0.0007) -[2023-10-15 18:33:09,253][52866] Updated weights for policy 1, policy_version 94700 (0.0009) -[2023-10-15 18:33:09,389][52833] Updated weights for policy 0, policy_version 94460 (0.0008) -[2023-10-15 18:33:09,623][52866] Updated weights for policy 1, policy_version 94710 (0.0008) -[2023-10-15 18:33:09,994][52866] Updated weights for policy 1, policy_version 94720 (0.0008) -[2023-10-15 18:33:13,227][52833] Updated weights for policy 0, policy_version 94470 (0.0008) -[2023-10-15 18:33:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 193724416. Throughput: 0: 1807.9, 1: 1792.8. Samples: 48449118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:13,441][51532] Avg episode reward: [(0, '72.480'), (1, '60.910')] -[2023-10-15 18:33:13,603][52833] Updated weights for policy 0, policy_version 94480 (0.0008) -[2023-10-15 18:33:13,663][52866] Updated weights for policy 1, policy_version 94730 (0.0007) -[2023-10-15 18:33:13,968][52833] Updated weights for policy 0, policy_version 94490 (0.0010) -[2023-10-15 18:33:14,026][52866] Updated weights for policy 1, policy_version 94740 (0.0007) -[2023-10-15 18:33:14,401][52866] Updated weights for policy 1, policy_version 94750 (0.0007) -[2023-10-15 18:33:17,654][52833] Updated weights for policy 0, policy_version 94500 (0.0009) -[2023-10-15 18:33:18,026][52833] Updated weights for policy 0, policy_version 94510 (0.0007) -[2023-10-15 18:33:18,224][52866] Updated weights for policy 1, policy_version 94760 (0.0007) -[2023-10-15 18:33:18,387][52833] Updated weights for policy 0, policy_version 94520 (0.0007) -[2023-10-15 18:33:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 193789952. Throughput: 0: 1792.4, 1: 1777.6. Samples: 48458656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:18,441][51532] Avg episode reward: [(0, '71.860'), (1, '62.240')] -[2023-10-15 18:33:18,587][52866] Updated weights for policy 1, policy_version 94770 (0.0008) -[2023-10-15 18:33:18,948][52866] Updated weights for policy 1, policy_version 94780 (0.0008) -[2023-10-15 18:33:22,222][52833] Updated weights for policy 0, policy_version 94530 (0.0007) -[2023-10-15 18:33:22,596][52833] Updated weights for policy 0, policy_version 94540 (0.0008) -[2023-10-15 18:33:22,600][52866] Updated weights for policy 1, policy_version 94790 (0.0008) -[2023-10-15 18:33:22,957][52833] Updated weights for policy 0, policy_version 94550 (0.0009) -[2023-10-15 18:33:22,966][52866] Updated weights for policy 1, policy_version 94800 (0.0009) -[2023-10-15 18:33:23,326][52866] Updated weights for policy 1, policy_version 94810 (0.0008) -[2023-10-15 18:33:23,331][52833] Updated weights for policy 0, policy_version 94560 (0.0008) -[2023-10-15 18:33:23,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 193888256. Throughput: 0: 1798.1, 1: 1794.9. Samples: 48480934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:23,442][51532] Avg episode reward: [(0, '75.080'), (1, '62.760')] -[2023-10-15 18:33:26,955][52866] Updated weights for policy 1, policy_version 94820 (0.0008) -[2023-10-15 18:33:27,142][52833] Updated weights for policy 0, policy_version 94570 (0.0009) -[2023-10-15 18:33:27,318][52866] Updated weights for policy 1, policy_version 94830 (0.0007) -[2023-10-15 18:33:27,513][52833] Updated weights for policy 0, policy_version 94580 (0.0008) -[2023-10-15 18:33:27,672][52866] Updated weights for policy 1, policy_version 94840 (0.0007) -[2023-10-15 18:33:27,872][52833] Updated weights for policy 0, policy_version 94590 (0.0009) -[2023-10-15 18:33:28,441][51532] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 193986560. Throughput: 0: 1786.7, 1: 1783.2. Samples: 48500766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:28,442][51532] Avg episode reward: [(0, '75.380'), (1, '63.940')] -[2023-10-15 18:33:31,246][52866] Updated weights for policy 1, policy_version 94850 (0.0008) -[2023-10-15 18:33:31,609][52866] Updated weights for policy 1, policy_version 94860 (0.0009) -[2023-10-15 18:33:31,688][52833] Updated weights for policy 0, policy_version 94600 (0.0007) -[2023-10-15 18:33:31,968][52866] Updated weights for policy 1, policy_version 94870 (0.0009) -[2023-10-15 18:33:32,058][52833] Updated weights for policy 0, policy_version 94610 (0.0007) -[2023-10-15 18:33:32,333][52866] Updated weights for policy 1, policy_version 94880 (0.0008) -[2023-10-15 18:33:32,432][52833] Updated weights for policy 0, policy_version 94620 (0.0008) -[2023-10-15 18:33:33,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 194052096. Throughput: 0: 1792.3, 1: 1799.7. Samples: 48513410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:33,442][51532] Avg episode reward: [(0, '77.240'), (1, '65.810')] -[2023-10-15 18:33:36,088][52833] Updated weights for policy 0, policy_version 94630 (0.0008) -[2023-10-15 18:33:36,099][52866] Updated weights for policy 1, policy_version 94890 (0.0008) -[2023-10-15 18:33:36,449][52833] Updated weights for policy 0, policy_version 94640 (0.0009) -[2023-10-15 18:33:36,459][52866] Updated weights for policy 1, policy_version 94900 (0.0010) -[2023-10-15 18:33:36,815][52833] Updated weights for policy 0, policy_version 94650 (0.0008) -[2023-10-15 18:33:36,821][52866] Updated weights for policy 1, policy_version 94910 (0.0007) -[2023-10-15 18:33:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194117632. Throughput: 0: 1793.9, 1: 1792.3. Samples: 48533356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:38,441][51532] Avg episode reward: [(0, '75.760'), (1, '66.620')] -[2023-10-15 18:33:40,484][52833] Updated weights for policy 0, policy_version 94660 (0.0007) -[2023-10-15 18:33:40,626][52866] Updated weights for policy 1, policy_version 94920 (0.0009) -[2023-10-15 18:33:40,855][52833] Updated weights for policy 0, policy_version 94670 (0.0007) -[2023-10-15 18:33:40,988][52866] Updated weights for policy 1, policy_version 94930 (0.0008) -[2023-10-15 18:33:41,217][52833] Updated weights for policy 0, policy_version 94680 (0.0008) -[2023-10-15 18:33:41,361][52866] Updated weights for policy 1, policy_version 94940 (0.0009) -[2023-10-15 18:33:43,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194183168. Throughput: 0: 1791.4, 1: 1798.6. Samples: 48555836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:43,441][51532] Avg episode reward: [(0, '74.990'), (1, '65.410')] -[2023-10-15 18:33:43,452][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000094944_97222656.pth... -[2023-10-15 18:33:43,452][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000094688_96960512.pth... -[2023-10-15 18:33:43,488][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000093024_95256576.pth -[2023-10-15 18:33:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000093280_95518720.pth -[2023-10-15 18:33:44,839][52833] Updated weights for policy 0, policy_version 94690 (0.0008) -[2023-10-15 18:33:45,140][52866] Updated weights for policy 1, policy_version 94950 (0.0008) -[2023-10-15 18:33:45,200][52833] Updated weights for policy 0, policy_version 94700 (0.0008) -[2023-10-15 18:33:45,507][52866] Updated weights for policy 1, policy_version 94960 (0.0008) -[2023-10-15 18:33:45,565][52833] Updated weights for policy 0, policy_version 94710 (0.0008) -[2023-10-15 18:33:45,875][52866] Updated weights for policy 1, policy_version 94970 (0.0008) -[2023-10-15 18:33:45,936][52833] Updated weights for policy 0, policy_version 94720 (0.0009) -[2023-10-15 18:33:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194248704. Throughput: 0: 1791.8, 1: 1804.4. Samples: 48566052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:48,441][51532] Avg episode reward: [(0, '74.360'), (1, '67.070')] -[2023-10-15 18:33:49,741][52833] Updated weights for policy 0, policy_version 94730 (0.0009) -[2023-10-15 18:33:49,808][52866] Updated weights for policy 1, policy_version 94980 (0.0010) -[2023-10-15 18:33:50,102][52833] Updated weights for policy 0, policy_version 94740 (0.0007) -[2023-10-15 18:33:50,168][52866] Updated weights for policy 1, policy_version 94990 (0.0008) -[2023-10-15 18:33:50,470][52833] Updated weights for policy 0, policy_version 94750 (0.0007) -[2023-10-15 18:33:50,539][52866] Updated weights for policy 1, policy_version 95000 (0.0007) -[2023-10-15 18:33:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 194314240. Throughput: 0: 1792.7, 1: 1797.3. Samples: 48588160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:33:53,442][51532] Avg episode reward: [(0, '72.340'), (1, '70.150')] -[2023-10-15 18:33:54,152][52833] Updated weights for policy 0, policy_version 94760 (0.0008) -[2023-10-15 18:33:54,236][52866] Updated weights for policy 1, policy_version 95010 (0.0008) -[2023-10-15 18:33:54,515][52833] Updated weights for policy 0, policy_version 94770 (0.0007) -[2023-10-15 18:33:54,597][52866] Updated weights for policy 1, policy_version 95020 (0.0007) -[2023-10-15 18:33:54,889][52833] Updated weights for policy 0, policy_version 94780 (0.0008) -[2023-10-15 18:33:54,962][52866] Updated weights for policy 1, policy_version 95030 (0.0008) -[2023-10-15 18:33:55,330][52866] Updated weights for policy 1, policy_version 95040 (0.0008) -[2023-10-15 18:33:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 194379776. Throughput: 0: 1795.6, 1: 1792.2. Samples: 48610570. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:33:58,442][51532] Avg episode reward: [(0, '75.770'), (1, '69.030')] -[2023-10-15 18:33:58,624][52833] Updated weights for policy 0, policy_version 94790 (0.0007) -[2023-10-15 18:33:58,991][52833] Updated weights for policy 0, policy_version 94800 (0.0008) -[2023-10-15 18:33:59,104][52866] Updated weights for policy 1, policy_version 95050 (0.0008) -[2023-10-15 18:33:59,350][52833] Updated weights for policy 0, policy_version 94810 (0.0009) -[2023-10-15 18:33:59,470][52866] Updated weights for policy 1, policy_version 95060 (0.0007) -[2023-10-15 18:33:59,836][52866] Updated weights for policy 1, policy_version 95070 (0.0008) -[2023-10-15 18:34:03,190][52833] Updated weights for policy 0, policy_version 94820 (0.0008) -[2023-10-15 18:34:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 194445312. Throughput: 0: 1798.0, 1: 1794.6. Samples: 48620326. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:03,442][51532] Avg episode reward: [(0, '78.590'), (1, '68.800')] -[2023-10-15 18:34:03,556][52833] Updated weights for policy 0, policy_version 94830 (0.0007) -[2023-10-15 18:34:03,639][52866] Updated weights for policy 1, policy_version 95080 (0.0007) -[2023-10-15 18:34:03,922][52833] Updated weights for policy 0, policy_version 94840 (0.0008) -[2023-10-15 18:34:04,008][52866] Updated weights for policy 1, policy_version 95090 (0.0007) -[2023-10-15 18:34:04,373][52866] Updated weights for policy 1, policy_version 95100 (0.0008) -[2023-10-15 18:34:07,748][52833] Updated weights for policy 0, policy_version 94850 (0.0009) -[2023-10-15 18:34:08,082][52866] Updated weights for policy 1, policy_version 95110 (0.0008) -[2023-10-15 18:34:08,109][52833] Updated weights for policy 0, policy_version 94860 (0.0008) -[2023-10-15 18:34:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 194510848. Throughput: 0: 1798.6, 1: 1797.2. Samples: 48642744. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:08,442][51532] Avg episode reward: [(0, '81.110'), (1, '67.490')] -[2023-10-15 18:34:08,447][52866] Updated weights for policy 1, policy_version 95120 (0.0008) -[2023-10-15 18:34:08,486][52833] Updated weights for policy 0, policy_version 94870 (0.0010) -[2023-10-15 18:34:08,817][52866] Updated weights for policy 1, policy_version 95130 (0.0007) -[2023-10-15 18:34:08,845][52833] Updated weights for policy 0, policy_version 94880 (0.0007) -[2023-10-15 18:34:12,627][52833] Updated weights for policy 0, policy_version 94890 (0.0008) -[2023-10-15 18:34:12,648][52866] Updated weights for policy 1, policy_version 95140 (0.0010) -[2023-10-15 18:34:12,992][52833] Updated weights for policy 0, policy_version 94900 (0.0009) -[2023-10-15 18:34:13,009][52866] Updated weights for policy 1, policy_version 95150 (0.0008) -[2023-10-15 18:34:13,361][52833] Updated weights for policy 0, policy_version 94910 (0.0009) -[2023-10-15 18:34:13,372][52866] Updated weights for policy 1, policy_version 95160 (0.0008) -[2023-10-15 18:34:13,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 194609152. Throughput: 0: 1810.9, 1: 1811.2. Samples: 48663760. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:13,442][51532] Avg episode reward: [(0, '82.030'), (1, '68.360')] -[2023-10-15 18:34:17,041][52833] Updated weights for policy 0, policy_version 94920 (0.0008) -[2023-10-15 18:34:17,259][52866] Updated weights for policy 1, policy_version 95170 (0.0007) -[2023-10-15 18:34:17,410][52833] Updated weights for policy 0, policy_version 94930 (0.0007) -[2023-10-15 18:34:17,623][52866] Updated weights for policy 1, policy_version 95180 (0.0008) -[2023-10-15 18:34:17,780][52833] Updated weights for policy 0, policy_version 94940 (0.0009) -[2023-10-15 18:34:17,992][52866] Updated weights for policy 1, policy_version 95190 (0.0009) -[2023-10-15 18:34:18,359][52866] Updated weights for policy 1, policy_version 95200 (0.0008) -[2023-10-15 18:34:18,441][51532] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 194707456. Throughput: 0: 1799.7, 1: 1788.0. Samples: 48674854. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:18,442][51532] Avg episode reward: [(0, '80.980'), (1, '65.540')] -[2023-10-15 18:34:21,582][52833] Updated weights for policy 0, policy_version 94950 (0.0008) -[2023-10-15 18:34:21,951][52833] Updated weights for policy 0, policy_version 94960 (0.0007) -[2023-10-15 18:34:22,013][52866] Updated weights for policy 1, policy_version 95210 (0.0008) -[2023-10-15 18:34:22,315][52833] Updated weights for policy 0, policy_version 94970 (0.0007) -[2023-10-15 18:34:22,378][52866] Updated weights for policy 1, policy_version 95220 (0.0008) -[2023-10-15 18:34:22,750][52866] Updated weights for policy 1, policy_version 95230 (0.0009) -[2023-10-15 18:34:23,441][51532] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 194772992. Throughput: 0: 1808.4, 1: 1810.5. Samples: 48696204. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:23,441][51532] Avg episode reward: [(0, '83.020'), (1, '63.100')] -[2023-10-15 18:34:25,984][52833] Updated weights for policy 0, policy_version 94980 (0.0008) -[2023-10-15 18:34:26,351][52833] Updated weights for policy 0, policy_version 94990 (0.0009) -[2023-10-15 18:34:26,623][52866] Updated weights for policy 1, policy_version 95240 (0.0008) -[2023-10-15 18:34:26,712][52833] Updated weights for policy 0, policy_version 95000 (0.0008) -[2023-10-15 18:34:26,979][52866] Updated weights for policy 1, policy_version 95250 (0.0009) -[2023-10-15 18:34:27,339][52866] Updated weights for policy 1, policy_version 95260 (0.0007) -[2023-10-15 18:34:28,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194838528. Throughput: 0: 1788.8, 1: 1786.1. Samples: 48716704. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:28,441][51532] Avg episode reward: [(0, '82.630'), (1, '60.800')] -[2023-10-15 18:34:30,455][52833] Updated weights for policy 0, policy_version 95010 (0.0007) -[2023-10-15 18:34:30,811][52833] Updated weights for policy 0, policy_version 95020 (0.0007) -[2023-10-15 18:34:31,133][52866] Updated weights for policy 1, policy_version 95270 (0.0008) -[2023-10-15 18:34:31,181][52833] Updated weights for policy 0, policy_version 95030 (0.0010) -[2023-10-15 18:34:31,499][52866] Updated weights for policy 1, policy_version 95280 (0.0009) -[2023-10-15 18:34:31,550][52833] Updated weights for policy 0, policy_version 95040 (0.0009) -[2023-10-15 18:34:31,863][52866] Updated weights for policy 1, policy_version 95290 (0.0007) -[2023-10-15 18:34:33,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194904064. Throughput: 0: 1805.7, 1: 1809.1. Samples: 48728720. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:33,442][51532] Avg episode reward: [(0, '82.600'), (1, '58.960')] -[2023-10-15 18:34:35,394][52833] Updated weights for policy 0, policy_version 95050 (0.0007) -[2023-10-15 18:34:35,630][52866] Updated weights for policy 1, policy_version 95300 (0.0008) -[2023-10-15 18:34:35,769][52833] Updated weights for policy 0, policy_version 95060 (0.0007) -[2023-10-15 18:34:35,995][52866] Updated weights for policy 1, policy_version 95310 (0.0010) -[2023-10-15 18:34:36,135][52833] Updated weights for policy 0, policy_version 95070 (0.0007) -[2023-10-15 18:34:36,364][52866] Updated weights for policy 1, policy_version 95320 (0.0010) -[2023-10-15 18:34:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194969600. Throughput: 0: 1784.5, 1: 1786.5. Samples: 48748856. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:38,441][51532] Avg episode reward: [(0, '81.310'), (1, '59.640')] -[2023-10-15 18:34:40,047][52866] Updated weights for policy 1, policy_version 95330 (0.0008) -[2023-10-15 18:34:40,084][52833] Updated weights for policy 0, policy_version 95080 (0.0008) -[2023-10-15 18:34:40,415][52866] Updated weights for policy 1, policy_version 95340 (0.0007) -[2023-10-15 18:34:40,451][52833] Updated weights for policy 0, policy_version 95090 (0.0007) -[2023-10-15 18:34:40,781][52866] Updated weights for policy 1, policy_version 95350 (0.0008) -[2023-10-15 18:34:40,823][52833] Updated weights for policy 0, policy_version 95100 (0.0007) -[2023-10-15 18:34:41,143][52866] Updated weights for policy 1, policy_version 95360 (0.0009) -[2023-10-15 18:34:43,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 195035136. Throughput: 0: 1778.5, 1: 1793.3. Samples: 48771300. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:43,441][51532] Avg episode reward: [(0, '82.370'), (1, '58.110')] -[2023-10-15 18:34:44,550][52833] Updated weights for policy 0, policy_version 95110 (0.0007) -[2023-10-15 18:34:44,714][52866] Updated weights for policy 1, policy_version 95370 (0.0008) -[2023-10-15 18:34:44,914][52833] Updated weights for policy 0, policy_version 95120 (0.0007) -[2023-10-15 18:34:45,072][52866] Updated weights for policy 1, policy_version 95380 (0.0009) -[2023-10-15 18:34:45,283][52833] Updated weights for policy 0, policy_version 95130 (0.0007) -[2023-10-15 18:34:45,450][52866] Updated weights for policy 1, policy_version 95390 (0.0009) -[2023-10-15 18:34:48,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 195100672. Throughput: 0: 1782.9, 1: 1791.1. Samples: 48781154. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:48,441][51532] Avg episode reward: [(0, '82.440'), (1, '58.060')] -[2023-10-15 18:34:49,047][52833] Updated weights for policy 0, policy_version 95140 (0.0008) -[2023-10-15 18:34:49,330][52866] Updated weights for policy 1, policy_version 95400 (0.0008) -[2023-10-15 18:34:49,417][52833] Updated weights for policy 0, policy_version 95150 (0.0009) -[2023-10-15 18:34:49,701][52866] Updated weights for policy 1, policy_version 95410 (0.0008) -[2023-10-15 18:34:49,779][52833] Updated weights for policy 0, policy_version 95160 (0.0008) -[2023-10-15 18:34:50,068][52866] Updated weights for policy 1, policy_version 95420 (0.0009) -[2023-10-15 18:34:53,435][52833] Updated weights for policy 0, policy_version 95170 (0.0009) -[2023-10-15 18:34:53,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 195166208. Throughput: 0: 1785.2, 1: 1788.1. Samples: 48803542. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) -[2023-10-15 18:34:53,441][51532] Avg episode reward: [(0, '81.750'), (1, '58.400')] -[2023-10-15 18:34:53,780][52866] Updated weights for policy 1, policy_version 95430 (0.0007) -[2023-10-15 18:34:53,803][52833] Updated weights for policy 0, policy_version 95180 (0.0008) -[2023-10-15 18:34:54,145][52866] Updated weights for policy 1, policy_version 95440 (0.0008) -[2023-10-15 18:34:54,167][52833] Updated weights for policy 0, policy_version 95190 (0.0007) -[2023-10-15 18:34:54,503][52866] Updated weights for policy 1, policy_version 95450 (0.0011) -[2023-10-15 18:34:54,535][52833] Updated weights for policy 0, policy_version 95200 (0.0009) -[2023-10-15 18:34:58,254][52866] Updated weights for policy 1, policy_version 95460 (0.0008) -[2023-10-15 18:34:58,299][52833] Updated weights for policy 0, policy_version 95210 (0.0007) -[2023-10-15 18:34:58,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 195231744. Throughput: 0: 1801.5, 1: 1806.7. Samples: 48826128. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:34:58,442][51532] Avg episode reward: [(0, '83.440'), (1, '60.710')] -[2023-10-15 18:34:58,621][52866] Updated weights for policy 1, policy_version 95470 (0.0008) -[2023-10-15 18:34:58,661][52833] Updated weights for policy 0, policy_version 95220 (0.0007) -[2023-10-15 18:34:58,976][52866] Updated weights for policy 1, policy_version 95480 (0.0008) -[2023-10-15 18:34:59,033][52833] Updated weights for policy 0, policy_version 95230 (0.0007) -[2023-10-15 18:35:02,757][52866] Updated weights for policy 1, policy_version 95490 (0.0008) -[2023-10-15 18:35:03,001][52833] Updated weights for policy 0, policy_version 95240 (0.0007) -[2023-10-15 18:35:03,126][52866] Updated weights for policy 1, policy_version 95500 (0.0007) -[2023-10-15 18:35:03,375][52833] Updated weights for policy 0, policy_version 95250 (0.0008) -[2023-10-15 18:35:03,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 195297280. Throughput: 0: 1784.3, 1: 1795.1. Samples: 48835924. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:03,442][51532] Avg episode reward: [(0, '82.670'), (1, '59.350')] -[2023-10-15 18:35:03,498][52866] Updated weights for policy 1, policy_version 95510 (0.0008) -[2023-10-15 18:35:03,740][52833] Updated weights for policy 0, policy_version 95260 (0.0008) -[2023-10-15 18:35:03,862][52866] Updated weights for policy 1, policy_version 95520 (0.0009) -[2023-10-15 18:35:07,373][52833] Updated weights for policy 0, policy_version 95270 (0.0008) -[2023-10-15 18:35:07,597][52866] Updated weights for policy 1, policy_version 95530 (0.0008) -[2023-10-15 18:35:07,733][52833] Updated weights for policy 0, policy_version 95280 (0.0008) -[2023-10-15 18:35:07,969][52866] Updated weights for policy 1, policy_version 95540 (0.0008) -[2023-10-15 18:35:08,106][52833] Updated weights for policy 0, policy_version 95290 (0.0010) -[2023-10-15 18:35:08,323][52866] Updated weights for policy 1, policy_version 95550 (0.0007) -[2023-10-15 18:35:08,441][51532] Fps is (10 sec: 19661.3, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 195428352. Throughput: 0: 1799.2, 1: 1802.3. Samples: 48858272. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:08,442][51532] Avg episode reward: [(0, '83.250'), (1, '61.610')] -[2023-10-15 18:35:11,790][52833] Updated weights for policy 0, policy_version 95300 (0.0008) -[2023-10-15 18:35:11,969][52866] Updated weights for policy 1, policy_version 95560 (0.0008) -[2023-10-15 18:35:12,164][52833] Updated weights for policy 0, policy_version 95310 (0.0007) -[2023-10-15 18:35:12,345][52866] Updated weights for policy 1, policy_version 95570 (0.0008) -[2023-10-15 18:35:12,524][52833] Updated weights for policy 0, policy_version 95320 (0.0008) -[2023-10-15 18:35:12,709][52866] Updated weights for policy 1, policy_version 95580 (0.0010) -[2023-10-15 18:35:13,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 195493888. Throughput: 0: 1787.3, 1: 1793.4. Samples: 48877838. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:13,442][51532] Avg episode reward: [(0, '83.850'), (1, '62.780')] -[2023-10-15 18:35:16,244][52833] Updated weights for policy 0, policy_version 95330 (0.0007) -[2023-10-15 18:35:16,501][52866] Updated weights for policy 1, policy_version 95590 (0.0009) -[2023-10-15 18:35:16,622][52833] Updated weights for policy 0, policy_version 95340 (0.0010) -[2023-10-15 18:35:16,865][52866] Updated weights for policy 1, policy_version 95600 (0.0008) -[2023-10-15 18:35:16,987][52833] Updated weights for policy 0, policy_version 95350 (0.0010) -[2023-10-15 18:35:17,228][52866] Updated weights for policy 1, policy_version 95610 (0.0007) -[2023-10-15 18:35:17,346][52833] Updated weights for policy 0, policy_version 95360 (0.0007) -[2023-10-15 18:35:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 195559424. Throughput: 0: 1800.4, 1: 1798.0. Samples: 48890652. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:18,442][51532] Avg episode reward: [(0, '81.500'), (1, '62.220')] -[2023-10-15 18:35:20,921][52866] Updated weights for policy 1, policy_version 95620 (0.0008) -[2023-10-15 18:35:21,115][52833] Updated weights for policy 0, policy_version 95370 (0.0008) -[2023-10-15 18:35:21,294][52866] Updated weights for policy 1, policy_version 95630 (0.0008) -[2023-10-15 18:35:21,475][52833] Updated weights for policy 0, policy_version 95380 (0.0009) -[2023-10-15 18:35:21,651][52866] Updated weights for policy 1, policy_version 95640 (0.0007) -[2023-10-15 18:35:21,847][52833] Updated weights for policy 0, policy_version 95390 (0.0008) -[2023-10-15 18:35:23,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 195624960. Throughput: 0: 1789.9, 1: 1796.3. Samples: 48910234. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:23,442][51532] Avg episode reward: [(0, '80.080'), (1, '61.780')] -[2023-10-15 18:35:25,422][52866] Updated weights for policy 1, policy_version 95650 (0.0008) -[2023-10-15 18:35:25,583][52833] Updated weights for policy 0, policy_version 95400 (0.0009) -[2023-10-15 18:35:25,795][52866] Updated weights for policy 1, policy_version 95660 (0.0008) -[2023-10-15 18:35:25,942][52833] Updated weights for policy 0, policy_version 95410 (0.0009) -[2023-10-15 18:35:26,159][52866] Updated weights for policy 1, policy_version 95670 (0.0011) -[2023-10-15 18:35:26,320][52833] Updated weights for policy 0, policy_version 95420 (0.0008) -[2023-10-15 18:35:26,524][52866] Updated weights for policy 1, policy_version 95680 (0.0007) -[2023-10-15 18:35:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 195690496. Throughput: 0: 1789.7, 1: 1792.6. Samples: 48932502. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:28,442][51532] Avg episode reward: [(0, '77.350'), (1, '62.890')] -[2023-10-15 18:35:30,186][52833] Updated weights for policy 0, policy_version 95430 (0.0008) -[2023-10-15 18:35:30,311][52866] Updated weights for policy 1, policy_version 95690 (0.0008) -[2023-10-15 18:35:30,565][52833] Updated weights for policy 0, policy_version 95440 (0.0008) -[2023-10-15 18:35:30,678][52866] Updated weights for policy 1, policy_version 95700 (0.0008) -[2023-10-15 18:35:30,926][52833] Updated weights for policy 0, policy_version 95450 (0.0008) -[2023-10-15 18:35:31,044][52866] Updated weights for policy 1, policy_version 95710 (0.0009) -[2023-10-15 18:35:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 195756032. Throughput: 0: 1794.6, 1: 1798.2. Samples: 48942828. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:33,442][51532] Avg episode reward: [(0, '80.110'), (1, '64.490')] -[2023-10-15 18:35:34,741][52833] Updated weights for policy 0, policy_version 95460 (0.0007) -[2023-10-15 18:35:34,893][52866] Updated weights for policy 1, policy_version 95720 (0.0010) -[2023-10-15 18:35:35,114][52833] Updated weights for policy 0, policy_version 95470 (0.0008) -[2023-10-15 18:35:35,266][52866] Updated weights for policy 1, policy_version 95730 (0.0008) -[2023-10-15 18:35:35,477][52833] Updated weights for policy 0, policy_version 95480 (0.0008) -[2023-10-15 18:35:35,624][52866] Updated weights for policy 1, policy_version 95740 (0.0008) -[2023-10-15 18:35:38,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 195821568. Throughput: 0: 1783.6, 1: 1788.1. Samples: 48964272. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:38,442][51532] Avg episode reward: [(0, '80.990'), (1, '63.500')] -[2023-10-15 18:35:39,237][52833] Updated weights for policy 0, policy_version 95490 (0.0008) -[2023-10-15 18:35:39,468][52866] Updated weights for policy 1, policy_version 95750 (0.0007) -[2023-10-15 18:35:39,603][52833] Updated weights for policy 0, policy_version 95500 (0.0009) -[2023-10-15 18:35:39,831][52866] Updated weights for policy 1, policy_version 95760 (0.0009) -[2023-10-15 18:35:39,975][52833] Updated weights for policy 0, policy_version 95510 (0.0007) -[2023-10-15 18:35:40,198][52866] Updated weights for policy 1, policy_version 95770 (0.0009) -[2023-10-15 18:35:40,344][52833] Updated weights for policy 0, policy_version 95520 (0.0008) -[2023-10-15 18:35:43,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 195887104. Throughput: 0: 1782.6, 1: 1778.1. Samples: 48986358. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:43,442][51532] Avg episode reward: [(0, '79.820'), (1, '66.370')] -[2023-10-15 18:35:43,454][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000095520_97812480.pth... -[2023-10-15 18:35:43,454][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000095776_98074624.pth... -[2023-10-15 18:35:43,491][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000094112_96370688.pth -[2023-10-15 18:35:43,494][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000093856_96108544.pth -[2023-10-15 18:35:44,046][52866] Updated weights for policy 1, policy_version 95780 (0.0009) -[2023-10-15 18:35:44,076][52833] Updated weights for policy 0, policy_version 95530 (0.0008) -[2023-10-15 18:35:44,422][52866] Updated weights for policy 1, policy_version 95790 (0.0009) -[2023-10-15 18:35:44,440][52833] Updated weights for policy 0, policy_version 95540 (0.0007) -[2023-10-15 18:35:44,784][52866] Updated weights for policy 1, policy_version 95800 (0.0008) -[2023-10-15 18:35:44,814][52833] Updated weights for policy 0, policy_version 95550 (0.0008) -[2023-10-15 18:35:48,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 195952640. Throughput: 0: 1789.4, 1: 1777.2. Samples: 48996424. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:48,442][51532] Avg episode reward: [(0, '77.250'), (1, '66.840')] -[2023-10-15 18:35:48,550][52833] Updated weights for policy 0, policy_version 95560 (0.0009) -[2023-10-15 18:35:48,679][52866] Updated weights for policy 1, policy_version 95810 (0.0007) -[2023-10-15 18:35:48,917][52833] Updated weights for policy 0, policy_version 95570 (0.0009) -[2023-10-15 18:35:49,052][52866] Updated weights for policy 1, policy_version 95820 (0.0008) -[2023-10-15 18:35:49,281][52833] Updated weights for policy 0, policy_version 95580 (0.0008) -[2023-10-15 18:35:49,411][52866] Updated weights for policy 1, policy_version 95830 (0.0007) -[2023-10-15 18:35:49,783][52866] Updated weights for policy 1, policy_version 95840 (0.0010) -[2023-10-15 18:35:53,135][52833] Updated weights for policy 0, policy_version 95590 (0.0007) -[2023-10-15 18:35:53,441][51532] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 196018176. Throughput: 0: 1786.2, 1: 1772.2. Samples: 49018400. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) -[2023-10-15 18:35:53,441][51532] Avg episode reward: [(0, '76.600'), (1, '68.360')] -[2023-10-15 18:35:53,500][52833] Updated weights for policy 0, policy_version 95600 (0.0008) -[2023-10-15 18:35:53,597][52866] Updated weights for policy 1, policy_version 95850 (0.0008) -[2023-10-15 18:35:53,868][52833] Updated weights for policy 0, policy_version 95610 (0.0007) -[2023-10-15 18:35:53,967][52866] Updated weights for policy 1, policy_version 95860 (0.0007) -[2023-10-15 18:35:54,334][52866] Updated weights for policy 1, policy_version 95870 (0.0009) -[2023-10-15 18:35:57,603][52833] Updated weights for policy 0, policy_version 95620 (0.0008) -[2023-10-15 18:35:57,970][52833] Updated weights for policy 0, policy_version 95630 (0.0009) -[2023-10-15 18:35:58,184][52866] Updated weights for policy 1, policy_version 95880 (0.0007) -[2023-10-15 18:35:58,330][52833] Updated weights for policy 0, policy_version 95640 (0.0009) -[2023-10-15 18:35:58,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 196083712. Throughput: 0: 1806.7, 1: 1796.9. Samples: 49040000. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:35:58,441][51532] Avg episode reward: [(0, '78.690'), (1, '68.450')] -[2023-10-15 18:35:58,551][52866] Updated weights for policy 1, policy_version 95890 (0.0007) -[2023-10-15 18:35:58,917][52866] Updated weights for policy 1, policy_version 95900 (0.0011) -[2023-10-15 18:36:02,023][52833] Updated weights for policy 0, policy_version 95650 (0.0008) -[2023-10-15 18:36:02,394][52833] Updated weights for policy 0, policy_version 95660 (0.0008) -[2023-10-15 18:36:02,652][52866] Updated weights for policy 1, policy_version 95910 (0.0008) -[2023-10-15 18:36:02,758][52833] Updated weights for policy 0, policy_version 95670 (0.0008) -[2023-10-15 18:36:03,014][52866] Updated weights for policy 1, policy_version 95920 (0.0007) -[2023-10-15 18:36:03,123][52833] Updated weights for policy 0, policy_version 95680 (0.0008) -[2023-10-15 18:36:03,385][52866] Updated weights for policy 1, policy_version 95930 (0.0010) -[2023-10-15 18:36:03,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 196182016. Throughput: 0: 1783.3, 1: 1763.0. Samples: 49050236. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:03,442][51532] Avg episode reward: [(0, '78.730'), (1, '68.990')] -[2023-10-15 18:36:06,919][52833] Updated weights for policy 0, policy_version 95690 (0.0008) -[2023-10-15 18:36:07,193][52866] Updated weights for policy 1, policy_version 95940 (0.0009) -[2023-10-15 18:36:07,293][52833] Updated weights for policy 0, policy_version 95700 (0.0007) -[2023-10-15 18:36:07,559][52866] Updated weights for policy 1, policy_version 95950 (0.0009) -[2023-10-15 18:36:07,653][52833] Updated weights for policy 0, policy_version 95710 (0.0008) -[2023-10-15 18:36:07,917][52866] Updated weights for policy 1, policy_version 95960 (0.0007) -[2023-10-15 18:36:08,441][51532] Fps is (10 sec: 19660.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196280320. Throughput: 0: 1803.5, 1: 1796.5. Samples: 49072234. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:08,441][51532] Avg episode reward: [(0, '79.210'), (1, '64.600')] -[2023-10-15 18:36:11,467][52833] Updated weights for policy 0, policy_version 95720 (0.0008) -[2023-10-15 18:36:11,624][52866] Updated weights for policy 1, policy_version 95970 (0.0008) -[2023-10-15 18:36:11,839][52833] Updated weights for policy 0, policy_version 95730 (0.0008) -[2023-10-15 18:36:11,992][52866] Updated weights for policy 1, policy_version 95980 (0.0007) -[2023-10-15 18:36:12,215][52833] Updated weights for policy 0, policy_version 95740 (0.0009) -[2023-10-15 18:36:12,368][52866] Updated weights for policy 1, policy_version 95990 (0.0009) -[2023-10-15 18:36:12,729][52866] Updated weights for policy 1, policy_version 96000 (0.0007) -[2023-10-15 18:36:13,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 196345856. Throughput: 0: 1783.6, 1: 1765.6. Samples: 49092218. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:13,442][51532] Avg episode reward: [(0, '79.230'), (1, '66.280')] -[2023-10-15 18:36:15,709][52833] Updated weights for policy 0, policy_version 95750 (0.0008) -[2023-10-15 18:36:16,074][52833] Updated weights for policy 0, policy_version 95760 (0.0009) -[2023-10-15 18:36:16,411][52866] Updated weights for policy 1, policy_version 96010 (0.0007) -[2023-10-15 18:36:16,448][52833] Updated weights for policy 0, policy_version 95770 (0.0008) -[2023-10-15 18:36:16,776][52866] Updated weights for policy 1, policy_version 96020 (0.0008) -[2023-10-15 18:36:17,144][52866] Updated weights for policy 1, policy_version 96030 (0.0008) -[2023-10-15 18:36:18,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196411392. Throughput: 0: 1803.5, 1: 1792.0. Samples: 49104624. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:18,441][51532] Avg episode reward: [(0, '77.190'), (1, '70.750')] -[2023-10-15 18:36:20,113][52833] Updated weights for policy 0, policy_version 95780 (0.0009) -[2023-10-15 18:36:20,486][52833] Updated weights for policy 0, policy_version 95790 (0.0007) -[2023-10-15 18:36:20,840][52866] Updated weights for policy 1, policy_version 96040 (0.0008) -[2023-10-15 18:36:20,857][52833] Updated weights for policy 0, policy_version 95800 (0.0007) -[2023-10-15 18:36:21,212][52866] Updated weights for policy 1, policy_version 96050 (0.0008) -[2023-10-15 18:36:21,574][52866] Updated weights for policy 1, policy_version 96060 (0.0009) -[2023-10-15 18:36:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196476928. Throughput: 0: 1799.5, 1: 1771.1. Samples: 49124948. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:23,442][51532] Avg episode reward: [(0, '84.070'), (1, '73.680')] -[2023-10-15 18:36:24,718][52833] Updated weights for policy 0, policy_version 95810 (0.0007) -[2023-10-15 18:36:25,082][52833] Updated weights for policy 0, policy_version 95820 (0.0008) -[2023-10-15 18:36:25,245][52866] Updated weights for policy 1, policy_version 96070 (0.0008) -[2023-10-15 18:36:25,443][52833] Updated weights for policy 0, policy_version 95830 (0.0007) -[2023-10-15 18:36:25,616][52866] Updated weights for policy 1, policy_version 96080 (0.0007) -[2023-10-15 18:36:25,807][52833] Updated weights for policy 0, policy_version 95840 (0.0007) -[2023-10-15 18:36:25,980][52866] Updated weights for policy 1, policy_version 96090 (0.0009) -[2023-10-15 18:36:28,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196542464. Throughput: 0: 1801.9, 1: 1779.6. Samples: 49147522. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:28,442][51532] Avg episode reward: [(0, '83.760'), (1, '76.530')] -[2023-10-15 18:36:29,567][52833] Updated weights for policy 0, policy_version 95850 (0.0007) -[2023-10-15 18:36:29,746][52866] Updated weights for policy 1, policy_version 96100 (0.0009) -[2023-10-15 18:36:29,934][52833] Updated weights for policy 0, policy_version 95860 (0.0009) -[2023-10-15 18:36:30,108][52866] Updated weights for policy 1, policy_version 96110 (0.0009) -[2023-10-15 18:36:30,299][52833] Updated weights for policy 0, policy_version 95870 (0.0008) -[2023-10-15 18:36:30,481][52866] Updated weights for policy 1, policy_version 96120 (0.0010) -[2023-10-15 18:36:33,441][51532] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 196608000. Throughput: 0: 1797.5, 1: 1782.4. Samples: 49157524. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:33,442][51532] Avg episode reward: [(0, '84.660'), (1, '76.580')] -[2023-10-15 18:36:34,168][52833] Updated weights for policy 0, policy_version 95880 (0.0008) -[2023-10-15 18:36:34,228][52866] Updated weights for policy 1, policy_version 96130 (0.0010) -[2023-10-15 18:36:34,532][52833] Updated weights for policy 0, policy_version 95890 (0.0008) -[2023-10-15 18:36:34,592][52866] Updated weights for policy 1, policy_version 96140 (0.0007) -[2023-10-15 18:36:34,904][52833] Updated weights for policy 0, policy_version 95900 (0.0008) -[2023-10-15 18:36:34,957][52866] Updated weights for policy 1, policy_version 96150 (0.0010) -[2023-10-15 18:36:35,325][52866] Updated weights for policy 1, policy_version 96160 (0.0010) -[2023-10-15 18:36:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 196673536. Throughput: 0: 1796.1, 1: 1787.7. Samples: 49179672. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:38,442][51532] Avg episode reward: [(0, '85.650'), (1, '73.970')] -[2023-10-15 18:36:38,715][52833] Updated weights for policy 0, policy_version 95910 (0.0007) -[2023-10-15 18:36:39,089][52833] Updated weights for policy 0, policy_version 95920 (0.0007) -[2023-10-15 18:36:39,192][52866] Updated weights for policy 1, policy_version 96170 (0.0008) -[2023-10-15 18:36:39,454][52833] Updated weights for policy 0, policy_version 95930 (0.0008) -[2023-10-15 18:36:39,563][52866] Updated weights for policy 1, policy_version 96180 (0.0008) -[2023-10-15 18:36:39,925][52866] Updated weights for policy 1, policy_version 96190 (0.0008) -[2023-10-15 18:36:43,252][52833] Updated weights for policy 0, policy_version 95940 (0.0007) -[2023-10-15 18:36:43,441][51532] Fps is (10 sec: 13107.8, 60 sec: 14199.6, 300 sec: 14218.0). Total num frames: 196739072. Throughput: 0: 1806.5, 1: 1797.4. Samples: 49202174. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:43,441][51532] Avg episode reward: [(0, '87.140'), (1, '75.540')] -[2023-10-15 18:36:43,610][52833] Updated weights for policy 0, policy_version 95950 (0.0009) -[2023-10-15 18:36:43,732][52866] Updated weights for policy 1, policy_version 96200 (0.0008) -[2023-10-15 18:36:43,987][52833] Updated weights for policy 0, policy_version 95960 (0.0009) -[2023-10-15 18:36:44,108][52866] Updated weights for policy 1, policy_version 96210 (0.0008) -[2023-10-15 18:36:44,276][52410] Saving new best policy, reward=87.140! -[2023-10-15 18:36:44,474][52866] Updated weights for policy 1, policy_version 96220 (0.0011) -[2023-10-15 18:36:47,669][52833] Updated weights for policy 0, policy_version 95970 (0.0010) -[2023-10-15 18:36:48,043][52833] Updated weights for policy 0, policy_version 95980 (0.0008) -[2023-10-15 18:36:48,204][52866] Updated weights for policy 1, policy_version 96230 (0.0008) -[2023-10-15 18:36:48,420][52833] Updated weights for policy 0, policy_version 95990 (0.0008) -[2023-10-15 18:36:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 196804608. Throughput: 0: 1792.8, 1: 1795.4. Samples: 49211708. Policy #0 lag: (min: 20.0, avg: 20.7, max: 38.0) -[2023-10-15 18:36:48,441][51532] Avg episode reward: [(0, '89.050'), (1, '73.750')] -[2023-10-15 18:36:48,569][52866] Updated weights for policy 1, policy_version 96240 (0.0008) -[2023-10-15 18:36:48,779][52410] Saving new best policy, reward=89.050! -[2023-10-15 18:36:48,781][52833] Updated weights for policy 0, policy_version 96000 (0.0009) -[2023-10-15 18:36:48,936][52866] Updated weights for policy 1, policy_version 96250 (0.0009) -[2023-10-15 18:36:52,554][52833] Updated weights for policy 0, policy_version 96010 (0.0007) -[2023-10-15 18:36:52,911][52866] Updated weights for policy 1, policy_version 96260 (0.0007) -[2023-10-15 18:36:52,919][52833] Updated weights for policy 0, policy_version 96020 (0.0007) -[2023-10-15 18:36:53,269][52866] Updated weights for policy 1, policy_version 96270 (0.0009) -[2023-10-15 18:36:53,289][52833] Updated weights for policy 0, policy_version 96030 (0.0008) -[2023-10-15 18:36:53,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 196902912. Throughput: 0: 1803.1, 1: 1787.5. Samples: 49233812. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:36:53,442][51532] Avg episode reward: [(0, '87.580'), (1, '71.920')] -[2023-10-15 18:36:53,644][52866] Updated weights for policy 1, policy_version 96280 (0.0010) -[2023-10-15 18:36:56,918][52833] Updated weights for policy 0, policy_version 96040 (0.0010) -[2023-10-15 18:36:57,277][52833] Updated weights for policy 0, policy_version 96050 (0.0010) -[2023-10-15 18:36:57,492][52866] Updated weights for policy 1, policy_version 96290 (0.0010) -[2023-10-15 18:36:57,637][52833] Updated weights for policy 0, policy_version 96060 (0.0008) -[2023-10-15 18:36:57,847][52866] Updated weights for policy 1, policy_version 96300 (0.0009) -[2023-10-15 18:36:58,222][52866] Updated weights for policy 1, policy_version 96310 (0.0009) -[2023-10-15 18:36:58,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 196968448. Throughput: 0: 1796.0, 1: 1798.8. Samples: 49253982. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:36:58,441][51532] Avg episode reward: [(0, '90.900'), (1, '70.170')] -[2023-10-15 18:36:58,450][52410] Saving new best policy, reward=90.900! -[2023-10-15 18:36:58,589][52866] Updated weights for policy 1, policy_version 96320 (0.0008) -[2023-10-15 18:37:01,348][52833] Updated weights for policy 0, policy_version 96070 (0.0009) -[2023-10-15 18:37:01,709][52833] Updated weights for policy 0, policy_version 96080 (0.0009) -[2023-10-15 18:37:02,081][52833] Updated weights for policy 0, policy_version 96090 (0.0010) -[2023-10-15 18:37:02,390][52866] Updated weights for policy 1, policy_version 96330 (0.0008) -[2023-10-15 18:37:02,755][52866] Updated weights for policy 1, policy_version 96340 (0.0009) -[2023-10-15 18:37:03,123][52866] Updated weights for policy 1, policy_version 96350 (0.0008) -[2023-10-15 18:37:03,441][51532] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 197066752. Throughput: 0: 1802.2, 1: 1782.5. Samples: 49265938. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:03,442][51532] Avg episode reward: [(0, '91.230'), (1, '70.770')] -[2023-10-15 18:37:03,442][52410] Saving new best policy, reward=91.230! -[2023-10-15 18:37:05,785][52833] Updated weights for policy 0, policy_version 96100 (0.0009) -[2023-10-15 18:37:06,156][52833] Updated weights for policy 0, policy_version 96110 (0.0009) -[2023-10-15 18:37:06,528][52833] Updated weights for policy 0, policy_version 96120 (0.0007) -[2023-10-15 18:37:06,865][52866] Updated weights for policy 1, policy_version 96360 (0.0008) -[2023-10-15 18:37:07,222][52866] Updated weights for policy 1, policy_version 96370 (0.0010) -[2023-10-15 18:37:07,588][52866] Updated weights for policy 1, policy_version 96380 (0.0012) -[2023-10-15 18:37:08,441][51532] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 197132288. Throughput: 0: 1787.0, 1: 1802.7. Samples: 49286486. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:08,442][51532] Avg episode reward: [(0, '93.120'), (1, '71.730')] -[2023-10-15 18:37:08,443][52410] Saving new best policy, reward=93.120! -[2023-10-15 18:37:10,323][52833] Updated weights for policy 0, policy_version 96130 (0.0008) -[2023-10-15 18:37:10,690][52833] Updated weights for policy 0, policy_version 96140 (0.0010) -[2023-10-15 18:37:11,053][52833] Updated weights for policy 0, policy_version 96150 (0.0010) -[2023-10-15 18:37:11,306][52866] Updated weights for policy 1, policy_version 96390 (0.0009) -[2023-10-15 18:37:11,420][52833] Updated weights for policy 0, policy_version 96160 (0.0008) -[2023-10-15 18:37:11,674][52866] Updated weights for policy 1, policy_version 96400 (0.0008) -[2023-10-15 18:37:12,050][52866] Updated weights for policy 1, policy_version 96410 (0.0010) -[2023-10-15 18:37:13,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 197197824. Throughput: 0: 1789.0, 1: 1778.6. Samples: 49308064. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:13,442][51532] Avg episode reward: [(0, '90.370'), (1, '70.590')] -[2023-10-15 18:37:15,110][52833] Updated weights for policy 0, policy_version 96170 (0.0007) -[2023-10-15 18:37:15,471][52833] Updated weights for policy 0, policy_version 96180 (0.0008) -[2023-10-15 18:37:15,719][52866] Updated weights for policy 1, policy_version 96420 (0.0009) -[2023-10-15 18:37:15,841][52833] Updated weights for policy 0, policy_version 96190 (0.0009) -[2023-10-15 18:37:16,086][52866] Updated weights for policy 1, policy_version 96430 (0.0010) -[2023-10-15 18:37:16,455][52866] Updated weights for policy 1, policy_version 96440 (0.0007) -[2023-10-15 18:37:18,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 197263360. Throughput: 0: 1790.4, 1: 1800.2. Samples: 49319100. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:18,441][51532] Avg episode reward: [(0, '88.500'), (1, '69.420')] -[2023-10-15 18:37:19,598][52833] Updated weights for policy 0, policy_version 96200 (0.0008) -[2023-10-15 18:37:19,975][52833] Updated weights for policy 0, policy_version 96210 (0.0009) -[2023-10-15 18:37:20,323][52866] Updated weights for policy 1, policy_version 96450 (0.0011) -[2023-10-15 18:37:20,347][52833] Updated weights for policy 0, policy_version 96220 (0.0008) -[2023-10-15 18:37:20,689][52866] Updated weights for policy 1, policy_version 96460 (0.0007) -[2023-10-15 18:37:21,054][52866] Updated weights for policy 1, policy_version 96470 (0.0008) -[2023-10-15 18:37:21,421][52866] Updated weights for policy 1, policy_version 96480 (0.0011) -[2023-10-15 18:37:23,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 197328896. Throughput: 0: 1790.5, 1: 1778.5. Samples: 49340278. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:23,441][51532] Avg episode reward: [(0, '85.830'), (1, '71.450')] -[2023-10-15 18:37:24,277][52833] Updated weights for policy 0, policy_version 96230 (0.0008) -[2023-10-15 18:37:24,660][52833] Updated weights for policy 0, policy_version 96240 (0.0009) -[2023-10-15 18:37:25,038][52833] Updated weights for policy 0, policy_version 96250 (0.0007) -[2023-10-15 18:37:25,240][52866] Updated weights for policy 1, policy_version 96490 (0.0007) -[2023-10-15 18:37:25,612][52866] Updated weights for policy 1, policy_version 96500 (0.0009) -[2023-10-15 18:37:25,980][52866] Updated weights for policy 1, policy_version 96510 (0.0009) -[2023-10-15 18:37:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 197394432. Throughput: 0: 1789.6, 1: 1775.9. Samples: 49362620. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:28,441][51532] Avg episode reward: [(0, '83.110'), (1, '68.620')] -[2023-10-15 18:37:28,738][52833] Updated weights for policy 0, policy_version 96260 (0.0007) -[2023-10-15 18:37:29,101][52833] Updated weights for policy 0, policy_version 96270 (0.0008) -[2023-10-15 18:37:29,477][52833] Updated weights for policy 0, policy_version 96280 (0.0007) -[2023-10-15 18:37:29,800][52866] Updated weights for policy 1, policy_version 96520 (0.0008) -[2023-10-15 18:37:30,164][52866] Updated weights for policy 1, policy_version 96530 (0.0010) -[2023-10-15 18:37:30,527][52866] Updated weights for policy 1, policy_version 96540 (0.0011) -[2023-10-15 18:37:33,244][52833] Updated weights for policy 0, policy_version 96290 (0.0007) -[2023-10-15 18:37:33,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14218.0). Total num frames: 197459968. Throughput: 0: 1793.9, 1: 1777.4. Samples: 49372418. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:33,441][51532] Avg episode reward: [(0, '80.320'), (1, '74.090')] -[2023-10-15 18:37:33,614][52833] Updated weights for policy 0, policy_version 96300 (0.0008) -[2023-10-15 18:37:33,989][52833] Updated weights for policy 0, policy_version 96310 (0.0009) -[2023-10-15 18:37:34,304][52866] Updated weights for policy 1, policy_version 96550 (0.0009) -[2023-10-15 18:37:34,369][52833] Updated weights for policy 0, policy_version 96320 (0.0010) -[2023-10-15 18:37:34,666][52866] Updated weights for policy 1, policy_version 96560 (0.0007) -[2023-10-15 18:37:35,036][52866] Updated weights for policy 1, policy_version 96570 (0.0007) -[2023-10-15 18:37:38,069][52833] Updated weights for policy 0, policy_version 96330 (0.0009) -[2023-10-15 18:37:38,440][52833] Updated weights for policy 0, policy_version 96340 (0.0008) -[2023-10-15 18:37:38,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 197525504. Throughput: 0: 1794.4, 1: 1779.8. Samples: 49394652. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:38,441][51532] Avg episode reward: [(0, '79.250'), (1, '74.100')] -[2023-10-15 18:37:38,805][52833] Updated weights for policy 0, policy_version 96350 (0.0009) -[2023-10-15 18:37:38,819][52866] Updated weights for policy 1, policy_version 96580 (0.0008) -[2023-10-15 18:37:39,179][52866] Updated weights for policy 1, policy_version 96590 (0.0007) -[2023-10-15 18:37:39,552][52866] Updated weights for policy 1, policy_version 96600 (0.0008) -[2023-10-15 18:37:42,698][52833] Updated weights for policy 0, policy_version 96360 (0.0007) -[2023-10-15 18:37:43,073][52833] Updated weights for policy 0, policy_version 96370 (0.0009) -[2023-10-15 18:37:43,152][52866] Updated weights for policy 1, policy_version 96610 (0.0008) -[2023-10-15 18:37:43,433][52833] Updated weights for policy 0, policy_version 96380 (0.0008) -[2023-10-15 18:37:43,441][51532] Fps is (10 sec: 13106.5, 60 sec: 14199.3, 300 sec: 14218.0). Total num frames: 197591040. Throughput: 0: 1811.9, 1: 1807.0. Samples: 49416834. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:43,443][51532] Avg episode reward: [(0, '75.990'), (1, '75.520')] -[2023-10-15 18:37:43,514][52866] Updated weights for policy 1, policy_version 96620 (0.0007) -[2023-10-15 18:37:43,578][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000096384_98697216.pth... -[2023-10-15 18:37:43,610][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000094688_96960512.pth -[2023-10-15 18:37:43,879][52866] Updated weights for policy 1, policy_version 96630 (0.0009) -[2023-10-15 18:37:44,239][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000096640_98959360.pth... -[2023-10-15 18:37:44,243][52866] Updated weights for policy 1, policy_version 96640 (0.0009) -[2023-10-15 18:37:44,268][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000094944_97222656.pth -[2023-10-15 18:37:47,120][52833] Updated weights for policy 0, policy_version 96390 (0.0008) -[2023-10-15 18:37:47,492][52833] Updated weights for policy 0, policy_version 96400 (0.0008) -[2023-10-15 18:37:47,858][52833] Updated weights for policy 0, policy_version 96410 (0.0007) -[2023-10-15 18:37:47,905][52866] Updated weights for policy 1, policy_version 96650 (0.0008) -[2023-10-15 18:37:48,269][52866] Updated weights for policy 1, policy_version 96660 (0.0007) -[2023-10-15 18:37:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 197689344. Throughput: 0: 1788.9, 1: 1791.5. Samples: 49427056. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-15 18:37:48,441][51532] Avg episode reward: [(0, '75.320'), (1, '74.780')] -[2023-10-15 18:37:48,634][52866] Updated weights for policy 1, policy_version 96670 (0.0008) -[2023-10-15 18:37:51,690][52833] Updated weights for policy 0, policy_version 96420 (0.0007) -[2023-10-15 18:37:52,066][52833] Updated weights for policy 0, policy_version 96430 (0.0007) -[2023-10-15 18:37:52,373][52866] Updated weights for policy 1, policy_version 96680 (0.0008) -[2023-10-15 18:37:52,432][52833] Updated weights for policy 0, policy_version 96440 (0.0008) -[2023-10-15 18:37:52,730][52866] Updated weights for policy 1, policy_version 96690 (0.0008) -[2023-10-15 18:37:53,104][52866] Updated weights for policy 1, policy_version 96700 (0.0008) -[2023-10-15 18:37:53,441][51532] Fps is (10 sec: 19661.9, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 197787648. Throughput: 0: 1806.5, 1: 1802.5. Samples: 49448890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:37:53,441][51532] Avg episode reward: [(0, '76.980'), (1, '76.870')] -[2023-10-15 18:37:55,953][52833] Updated weights for policy 0, policy_version 96450 (0.0008) -[2023-10-15 18:37:56,329][52833] Updated weights for policy 0, policy_version 96460 (0.0011) -[2023-10-15 18:37:56,694][52833] Updated weights for policy 0, policy_version 96470 (0.0012) -[2023-10-15 18:37:56,933][52866] Updated weights for policy 1, policy_version 96710 (0.0007) -[2023-10-15 18:37:57,062][52833] Updated weights for policy 0, policy_version 96480 (0.0007) -[2023-10-15 18:37:57,296][52866] Updated weights for policy 1, policy_version 96720 (0.0007) -[2023-10-15 18:37:57,661][52866] Updated weights for policy 1, policy_version 96730 (0.0009) -[2023-10-15 18:37:58,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 197853184. Throughput: 0: 1787.3, 1: 1790.1. Samples: 49469048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:37:58,442][51532] Avg episode reward: [(0, '74.310'), (1, '77.740')] -[2023-10-15 18:38:00,869][52833] Updated weights for policy 0, policy_version 96490 (0.0010) -[2023-10-15 18:38:01,235][52833] Updated weights for policy 0, policy_version 96500 (0.0008) -[2023-10-15 18:38:01,399][52866] Updated weights for policy 1, policy_version 96740 (0.0008) -[2023-10-15 18:38:01,593][52833] Updated weights for policy 0, policy_version 96510 (0.0010) -[2023-10-15 18:38:01,767][52866] Updated weights for policy 1, policy_version 96750 (0.0007) -[2023-10-15 18:38:02,130][52866] Updated weights for policy 1, policy_version 96760 (0.0008) -[2023-10-15 18:38:03,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 197918720. Throughput: 0: 1803.1, 1: 1802.4. Samples: 49481352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:03,442][51532] Avg episode reward: [(0, '77.620'), (1, '78.210')] -[2023-10-15 18:38:05,372][52833] Updated weights for policy 0, policy_version 96520 (0.0009) -[2023-10-15 18:38:05,743][52833] Updated weights for policy 0, policy_version 96530 (0.0011) -[2023-10-15 18:38:05,883][52866] Updated weights for policy 1, policy_version 96770 (0.0008) -[2023-10-15 18:38:06,104][52833] Updated weights for policy 0, policy_version 96540 (0.0007) -[2023-10-15 18:38:06,247][52866] Updated weights for policy 1, policy_version 96780 (0.0007) -[2023-10-15 18:38:06,616][52866] Updated weights for policy 1, policy_version 96790 (0.0008) -[2023-10-15 18:38:06,977][52866] Updated weights for policy 1, policy_version 96800 (0.0007) -[2023-10-15 18:38:08,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 197984256. Throughput: 0: 1788.3, 1: 1792.6. Samples: 49501416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:08,442][51532] Avg episode reward: [(0, '77.240'), (1, '76.020')] -[2023-10-15 18:38:09,779][52833] Updated weights for policy 0, policy_version 96550 (0.0008) -[2023-10-15 18:38:10,172][52833] Updated weights for policy 0, policy_version 96560 (0.0007) -[2023-10-15 18:38:10,531][52833] Updated weights for policy 0, policy_version 96570 (0.0007) -[2023-10-15 18:38:10,719][52866] Updated weights for policy 1, policy_version 96810 (0.0008) -[2023-10-15 18:38:11,091][52866] Updated weights for policy 1, policy_version 96820 (0.0010) -[2023-10-15 18:38:11,455][52866] Updated weights for policy 1, policy_version 96830 (0.0011) -[2023-10-15 18:38:13,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198049792. Throughput: 0: 1792.6, 1: 1792.2. Samples: 49523936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:13,441][51532] Avg episode reward: [(0, '75.000'), (1, '77.840')] -[2023-10-15 18:38:14,311][52833] Updated weights for policy 0, policy_version 96580 (0.0008) -[2023-10-15 18:38:14,680][52833] Updated weights for policy 0, policy_version 96590 (0.0010) -[2023-10-15 18:38:15,050][52833] Updated weights for policy 0, policy_version 96600 (0.0009) -[2023-10-15 18:38:15,285][52866] Updated weights for policy 1, policy_version 96840 (0.0008) -[2023-10-15 18:38:15,648][52866] Updated weights for policy 1, policy_version 96850 (0.0008) -[2023-10-15 18:38:16,015][52866] Updated weights for policy 1, policy_version 96860 (0.0010) -[2023-10-15 18:38:18,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 198115328. Throughput: 0: 1789.9, 1: 1799.6. Samples: 49533944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:18,442][51532] Avg episode reward: [(0, '76.810'), (1, '74.410')] -[2023-10-15 18:38:18,831][52833] Updated weights for policy 0, policy_version 96610 (0.0008) -[2023-10-15 18:38:19,195][52833] Updated weights for policy 0, policy_version 96620 (0.0011) -[2023-10-15 18:38:19,572][52833] Updated weights for policy 0, policy_version 96630 (0.0009) -[2023-10-15 18:38:19,750][52866] Updated weights for policy 1, policy_version 96870 (0.0009) -[2023-10-15 18:38:19,935][52833] Updated weights for policy 0, policy_version 96640 (0.0009) -[2023-10-15 18:38:20,120][52866] Updated weights for policy 1, policy_version 96880 (0.0010) -[2023-10-15 18:38:20,481][52866] Updated weights for policy 1, policy_version 96890 (0.0010) -[2023-10-15 18:38:23,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 198180864. Throughput: 0: 1785.4, 1: 1798.8. Samples: 49555940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:23,442][51532] Avg episode reward: [(0, '80.770'), (1, '75.170')] -[2023-10-15 18:38:23,662][52833] Updated weights for policy 0, policy_version 96650 (0.0008) -[2023-10-15 18:38:24,025][52833] Updated weights for policy 0, policy_version 96660 (0.0009) -[2023-10-15 18:38:24,173][52866] Updated weights for policy 1, policy_version 96900 (0.0009) -[2023-10-15 18:38:24,393][52833] Updated weights for policy 0, policy_version 96670 (0.0009) -[2023-10-15 18:38:24,531][52866] Updated weights for policy 1, policy_version 96910 (0.0008) -[2023-10-15 18:38:24,898][52866] Updated weights for policy 1, policy_version 96920 (0.0009) -[2023-10-15 18:38:28,228][52833] Updated weights for policy 0, policy_version 96680 (0.0010) -[2023-10-15 18:38:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 198246400. Throughput: 0: 1800.6, 1: 1790.7. Samples: 49578440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:28,441][51532] Avg episode reward: [(0, '81.090'), (1, '71.520')] -[2023-10-15 18:38:28,571][52866] Updated weights for policy 1, policy_version 96930 (0.0009) -[2023-10-15 18:38:28,598][52833] Updated weights for policy 0, policy_version 96690 (0.0009) -[2023-10-15 18:38:28,929][52866] Updated weights for policy 1, policy_version 96940 (0.0008) -[2023-10-15 18:38:28,970][52833] Updated weights for policy 0, policy_version 96700 (0.0008) -[2023-10-15 18:38:29,290][52866] Updated weights for policy 1, policy_version 96950 (0.0008) -[2023-10-15 18:38:29,654][52866] Updated weights for policy 1, policy_version 96960 (0.0010) -[2023-10-15 18:38:32,683][52833] Updated weights for policy 0, policy_version 96710 (0.0009) -[2023-10-15 18:38:33,040][52833] Updated weights for policy 0, policy_version 96720 (0.0008) -[2023-10-15 18:38:33,405][52833] Updated weights for policy 0, policy_version 96730 (0.0009) -[2023-10-15 18:38:33,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 198311936. Throughput: 0: 1788.0, 1: 1794.3. Samples: 49588258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:33,441][51532] Avg episode reward: [(0, '84.090'), (1, '65.710')] -[2023-10-15 18:38:33,477][52866] Updated weights for policy 1, policy_version 96970 (0.0008) -[2023-10-15 18:38:33,838][52866] Updated weights for policy 1, policy_version 96980 (0.0010) -[2023-10-15 18:38:34,200][52866] Updated weights for policy 1, policy_version 96990 (0.0011) -[2023-10-15 18:38:37,203][52833] Updated weights for policy 0, policy_version 96740 (0.0008) -[2023-10-15 18:38:37,571][52833] Updated weights for policy 0, policy_version 96750 (0.0008) -[2023-10-15 18:38:37,940][52833] Updated weights for policy 0, policy_version 96760 (0.0008) -[2023-10-15 18:38:37,987][52866] Updated weights for policy 1, policy_version 97000 (0.0007) -[2023-10-15 18:38:38,351][52866] Updated weights for policy 1, policy_version 97010 (0.0009) -[2023-10-15 18:38:38,441][51532] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 198410240. Throughput: 0: 1799.2, 1: 1792.9. Samples: 49610534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:38,441][51532] Avg episode reward: [(0, '88.650'), (1, '61.970')] -[2023-10-15 18:38:38,710][52866] Updated weights for policy 1, policy_version 97020 (0.0007) -[2023-10-15 18:38:41,620][52833] Updated weights for policy 0, policy_version 96770 (0.0008) -[2023-10-15 18:38:41,988][52833] Updated weights for policy 0, policy_version 96780 (0.0008) -[2023-10-15 18:38:42,364][52833] Updated weights for policy 0, policy_version 96790 (0.0008) -[2023-10-15 18:38:42,459][52866] Updated weights for policy 1, policy_version 97030 (0.0008) -[2023-10-15 18:38:42,723][52833] Updated weights for policy 0, policy_version 96800 (0.0008) -[2023-10-15 18:38:42,813][52866] Updated weights for policy 1, policy_version 97040 (0.0009) -[2023-10-15 18:38:43,186][52866] Updated weights for policy 1, policy_version 97050 (0.0010) -[2023-10-15 18:38:43,441][51532] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 198508544. Throughput: 0: 1786.5, 1: 1811.0. Samples: 49630938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:43,442][51532] Avg episode reward: [(0, '90.750'), (1, '61.720')] -[2023-10-15 18:38:46,389][52833] Updated weights for policy 0, policy_version 96810 (0.0007) -[2023-10-15 18:38:46,753][52833] Updated weights for policy 0, policy_version 96820 (0.0008) -[2023-10-15 18:38:46,969][52866] Updated weights for policy 1, policy_version 97060 (0.0007) -[2023-10-15 18:38:47,130][52833] Updated weights for policy 0, policy_version 96830 (0.0008) -[2023-10-15 18:38:47,331][52866] Updated weights for policy 1, policy_version 97070 (0.0007) -[2023-10-15 18:38:47,698][52866] Updated weights for policy 1, policy_version 97080 (0.0009) -[2023-10-15 18:38:48,441][51532] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 198574080. Throughput: 0: 1804.8, 1: 1793.7. Samples: 49643280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:48,442][51532] Avg episode reward: [(0, '90.520'), (1, '58.550')] -[2023-10-15 18:38:51,070][52833] Updated weights for policy 0, policy_version 96840 (0.0008) -[2023-10-15 18:38:51,386][52866] Updated weights for policy 1, policy_version 97090 (0.0010) -[2023-10-15 18:38:51,440][52833] Updated weights for policy 0, policy_version 96850 (0.0007) -[2023-10-15 18:38:51,749][52866] Updated weights for policy 1, policy_version 97100 (0.0008) -[2023-10-15 18:38:51,812][52833] Updated weights for policy 0, policy_version 96860 (0.0007) -[2023-10-15 18:38:52,119][52866] Updated weights for policy 1, policy_version 97110 (0.0007) -[2023-10-15 18:38:52,476][52866] Updated weights for policy 1, policy_version 97120 (0.0008) -[2023-10-15 18:38:53,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 198639616. Throughput: 0: 1790.7, 1: 1805.3. Samples: 49663236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-15 18:38:53,442][51532] Avg episode reward: [(0, '90.780'), (1, '55.630')] -[2023-10-15 18:38:55,723][52833] Updated weights for policy 0, policy_version 96870 (0.0007) -[2023-10-15 18:38:56,102][52833] Updated weights for policy 0, policy_version 96880 (0.0007) -[2023-10-15 18:38:56,340][52866] Updated weights for policy 1, policy_version 97130 (0.0007) -[2023-10-15 18:38:56,461][52833] Updated weights for policy 0, policy_version 96890 (0.0008) -[2023-10-15 18:38:56,717][52866] Updated weights for policy 1, policy_version 97140 (0.0009) -[2023-10-15 18:38:57,073][52866] Updated weights for policy 1, policy_version 97150 (0.0010) -[2023-10-15 18:38:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 198705152. Throughput: 0: 1777.1, 1: 1791.2. Samples: 49684510. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:38:58,442][51532] Avg episode reward: [(0, '93.100'), (1, '58.190')] -[2023-10-15 18:39:00,204][52833] Updated weights for policy 0, policy_version 96900 (0.0009) -[2023-10-15 18:39:00,575][52833] Updated weights for policy 0, policy_version 96910 (0.0009) -[2023-10-15 18:39:00,830][52866] Updated weights for policy 1, policy_version 97160 (0.0009) -[2023-10-15 18:39:00,942][52833] Updated weights for policy 0, policy_version 96920 (0.0007) -[2023-10-15 18:39:01,206][52866] Updated weights for policy 1, policy_version 97170 (0.0011) -[2023-10-15 18:39:01,574][52866] Updated weights for policy 1, policy_version 97180 (0.0007) -[2023-10-15 18:39:03,441][51532] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198770688. Throughput: 0: 1791.2, 1: 1804.1. Samples: 49695732. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:03,441][51532] Avg episode reward: [(0, '93.790'), (1, '61.930')] -[2023-10-15 18:39:03,442][52410] Saving new best policy, reward=93.790! -[2023-10-15 18:39:04,736][52833] Updated weights for policy 0, policy_version 96930 (0.0008) -[2023-10-15 18:39:05,093][52833] Updated weights for policy 0, policy_version 96940 (0.0009) -[2023-10-15 18:39:05,294][52866] Updated weights for policy 1, policy_version 97190 (0.0009) -[2023-10-15 18:39:05,465][52833] Updated weights for policy 0, policy_version 96950 (0.0008) -[2023-10-15 18:39:05,654][52866] Updated weights for policy 1, policy_version 97200 (0.0008) -[2023-10-15 18:39:05,828][52833] Updated weights for policy 0, policy_version 96960 (0.0008) -[2023-10-15 18:39:06,016][52866] Updated weights for policy 1, policy_version 97210 (0.0008) -[2023-10-15 18:39:08,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 198836224. Throughput: 0: 1787.3, 1: 1786.3. Samples: 49716752. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:08,442][51532] Avg episode reward: [(0, '93.470'), (1, '59.930')] -[2023-10-15 18:39:09,518][52833] Updated weights for policy 0, policy_version 96970 (0.0011) -[2023-10-15 18:39:09,889][52833] Updated weights for policy 0, policy_version 96980 (0.0008) -[2023-10-15 18:39:09,942][52866] Updated weights for policy 1, policy_version 97220 (0.0007) -[2023-10-15 18:39:10,259][52833] Updated weights for policy 0, policy_version 96990 (0.0007) -[2023-10-15 18:39:10,304][52866] Updated weights for policy 1, policy_version 97230 (0.0007) -[2023-10-15 18:39:10,667][52866] Updated weights for policy 1, policy_version 97240 (0.0007) -[2023-10-15 18:39:13,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 198901760. Throughput: 0: 1790.5, 1: 1790.1. Samples: 49739566. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:13,441][51532] Avg episode reward: [(0, '97.120'), (1, '60.040')] -[2023-10-15 18:39:13,452][52410] Saving new best policy, reward=97.120! -[2023-10-15 18:39:14,009][52833] Updated weights for policy 0, policy_version 97000 (0.0009) -[2023-10-15 18:39:14,387][52833] Updated weights for policy 0, policy_version 97010 (0.0011) -[2023-10-15 18:39:14,426][52866] Updated weights for policy 1, policy_version 97250 (0.0008) -[2023-10-15 18:39:14,754][52833] Updated weights for policy 0, policy_version 97020 (0.0009) -[2023-10-15 18:39:14,787][52866] Updated weights for policy 1, policy_version 97260 (0.0008) -[2023-10-15 18:39:15,161][52866] Updated weights for policy 1, policy_version 97270 (0.0010) -[2023-10-15 18:39:15,534][52866] Updated weights for policy 1, policy_version 97280 (0.0010) -[2023-10-15 18:39:18,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 198967296. Throughput: 0: 1791.9, 1: 1783.6. Samples: 49749158. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:18,442][51532] Avg episode reward: [(0, '98.440'), (1, '60.540')] -[2023-10-15 18:39:18,576][52833] Updated weights for policy 0, policy_version 97030 (0.0008) -[2023-10-15 18:39:18,943][52833] Updated weights for policy 0, policy_version 97040 (0.0007) -[2023-10-15 18:39:19,240][52866] Updated weights for policy 1, policy_version 97290 (0.0008) -[2023-10-15 18:39:19,317][52833] Updated weights for policy 0, policy_version 97050 (0.0009) -[2023-10-15 18:39:19,527][52410] Saving new best policy, reward=98.440! -[2023-10-15 18:39:19,601][52866] Updated weights for policy 1, policy_version 97300 (0.0007) -[2023-10-15 18:39:19,964][52866] Updated weights for policy 1, policy_version 97310 (0.0007) -[2023-10-15 18:39:23,028][52833] Updated weights for policy 0, policy_version 97060 (0.0009) -[2023-10-15 18:39:23,391][52833] Updated weights for policy 0, policy_version 97070 (0.0012) -[2023-10-15 18:39:23,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 199032832. Throughput: 0: 1791.2, 1: 1787.9. Samples: 49771592. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:23,442][51532] Avg episode reward: [(0, '98.640'), (1, '60.210')] -[2023-10-15 18:39:23,760][52833] Updated weights for policy 0, policy_version 97080 (0.0007) -[2023-10-15 18:39:23,777][52866] Updated weights for policy 1, policy_version 97320 (0.0007) -[2023-10-15 18:39:24,047][52410] Saving new best policy, reward=98.640! -[2023-10-15 18:39:24,139][52866] Updated weights for policy 1, policy_version 97330 (0.0008) -[2023-10-15 18:39:24,508][52866] Updated weights for policy 1, policy_version 97340 (0.0008) -[2023-10-15 18:39:27,491][52833] Updated weights for policy 0, policy_version 97090 (0.0009) -[2023-10-15 18:39:27,856][52833] Updated weights for policy 0, policy_version 97100 (0.0007) -[2023-10-15 18:39:28,211][52833] Updated weights for policy 0, policy_version 97110 (0.0007) -[2023-10-15 18:39:28,241][52866] Updated weights for policy 1, policy_version 97350 (0.0010) -[2023-10-15 18:39:28,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 199098368. Throughput: 0: 1807.1, 1: 1805.6. Samples: 49793510. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:28,442][51532] Avg episode reward: [(0, '97.120'), (1, '65.040')] -[2023-10-15 18:39:28,576][52833] Updated weights for policy 0, policy_version 97120 (0.0007) -[2023-10-15 18:39:28,604][52866] Updated weights for policy 1, policy_version 97360 (0.0007) -[2023-10-15 18:39:28,971][52866] Updated weights for policy 1, policy_version 97370 (0.0007) -[2023-10-15 18:39:32,281][52833] Updated weights for policy 0, policy_version 97130 (0.0008) -[2023-10-15 18:39:32,657][52833] Updated weights for policy 0, policy_version 97140 (0.0009) -[2023-10-15 18:39:32,698][52866] Updated weights for policy 1, policy_version 97380 (0.0008) -[2023-10-15 18:39:33,027][52833] Updated weights for policy 0, policy_version 97150 (0.0008) -[2023-10-15 18:39:33,060][52866] Updated weights for policy 1, policy_version 97390 (0.0007) -[2023-10-15 18:39:33,422][52866] Updated weights for policy 1, policy_version 97400 (0.0008) -[2023-10-15 18:39:33,441][51532] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 199196672. Throughput: 0: 1786.5, 1: 1785.2. Samples: 49804008. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:33,442][51532] Avg episode reward: [(0, '98.610'), (1, '64.510')] -[2023-10-15 18:39:36,976][52833] Updated weights for policy 0, policy_version 97160 (0.0010) -[2023-10-15 18:39:37,030][52866] Updated weights for policy 1, policy_version 97410 (0.0010) -[2023-10-15 18:39:37,342][52833] Updated weights for policy 0, policy_version 97170 (0.0009) -[2023-10-15 18:39:37,401][52866] Updated weights for policy 1, policy_version 97420 (0.0009) -[2023-10-15 18:39:37,719][52833] Updated weights for policy 0, policy_version 97180 (0.0011) -[2023-10-15 18:39:37,768][52866] Updated weights for policy 1, policy_version 97430 (0.0008) -[2023-10-15 18:39:38,133][52866] Updated weights for policy 1, policy_version 97440 (0.0011) -[2023-10-15 18:39:38,441][51532] Fps is (10 sec: 19661.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 199294976. Throughput: 0: 1809.7, 1: 1811.2. Samples: 49826176. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:38,442][51532] Avg episode reward: [(0, '99.040'), (1, '68.060')] -[2023-10-15 18:39:38,443][52410] Saving new best policy, reward=99.040! -[2023-10-15 18:39:41,589][52833] Updated weights for policy 0, policy_version 97190 (0.0007) -[2023-10-15 18:39:41,787][52866] Updated weights for policy 1, policy_version 97450 (0.0007) -[2023-10-15 18:39:41,966][52833] Updated weights for policy 0, policy_version 97200 (0.0007) -[2023-10-15 18:39:42,167][52866] Updated weights for policy 1, policy_version 97460 (0.0008) -[2023-10-15 18:39:42,346][52833] Updated weights for policy 0, policy_version 97210 (0.0008) -[2023-10-15 18:39:42,537][52866] Updated weights for policy 1, policy_version 97470 (0.0007) -[2023-10-15 18:39:43,441][51532] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 199360512. Throughput: 0: 1786.8, 1: 1797.9. Samples: 49845822. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:43,442][51532] Avg episode reward: [(0, '99.150'), (1, '66.710')] -[2023-10-15 18:39:43,456][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000097472_99811328.pth... -[2023-10-15 18:39:43,456][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000097216_99549184.pth... -[2023-10-15 18:39:43,490][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000095520_97812480.pth -[2023-10-15 18:39:43,493][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000095776_98074624.pth -[2023-10-15 18:39:43,494][52410] Saving new best policy, reward=99.150! -[2023-10-15 18:39:46,110][52833] Updated weights for policy 0, policy_version 97220 (0.0008) -[2023-10-15 18:39:46,430][52866] Updated weights for policy 1, policy_version 97480 (0.0008) -[2023-10-15 18:39:46,477][52833] Updated weights for policy 0, policy_version 97230 (0.0007) -[2023-10-15 18:39:46,794][52866] Updated weights for policy 1, policy_version 97490 (0.0007) -[2023-10-15 18:39:46,843][52833] Updated weights for policy 0, policy_version 97240 (0.0008) -[2023-10-15 18:39:47,156][52866] Updated weights for policy 1, policy_version 97500 (0.0008) -[2023-10-15 18:39:48,441][51532] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199426048. Throughput: 0: 1805.7, 1: 1810.0. Samples: 49858440. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:48,441][51532] Avg episode reward: [(0, '98.820'), (1, '67.810')] -[2023-10-15 18:39:50,536][52833] Updated weights for policy 0, policy_version 97250 (0.0007) -[2023-10-15 18:39:50,899][52833] Updated weights for policy 0, policy_version 97260 (0.0007) -[2023-10-15 18:39:50,928][52866] Updated weights for policy 1, policy_version 97510 (0.0009) -[2023-10-15 18:39:51,272][52833] Updated weights for policy 0, policy_version 97270 (0.0009) -[2023-10-15 18:39:51,297][52866] Updated weights for policy 1, policy_version 97520 (0.0009) -[2023-10-15 18:39:51,636][52833] Updated weights for policy 0, policy_version 97280 (0.0008) -[2023-10-15 18:39:51,658][52866] Updated weights for policy 1, policy_version 97530 (0.0008) -[2023-10-15 18:39:53,441][51532] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 199491584. Throughput: 0: 1783.0, 1: 1795.1. Samples: 49877770. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:53,442][51532] Avg episode reward: [(0, '97.590'), (1, '66.910')] -[2023-10-15 18:39:55,271][52866] Updated weights for policy 1, policy_version 97540 (0.0009) -[2023-10-15 18:39:55,310][52833] Updated weights for policy 0, policy_version 97290 (0.0008) -[2023-10-15 18:39:55,639][52866] Updated weights for policy 1, policy_version 97550 (0.0009) -[2023-10-15 18:39:55,669][52833] Updated weights for policy 0, policy_version 97300 (0.0007) -[2023-10-15 18:39:56,005][52866] Updated weights for policy 1, policy_version 97560 (0.0007) -[2023-10-15 18:39:56,035][52833] Updated weights for policy 0, policy_version 97310 (0.0007) -[2023-10-15 18:39:58,441][51532] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199557120. Throughput: 0: 1779.9, 1: 1796.4. Samples: 49900504. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) -[2023-10-15 18:39:58,442][51532] Avg episode reward: [(0, '94.600'), (1, '69.700')] -[2023-10-15 18:39:59,747][52833] Updated weights for policy 0, policy_version 97320 (0.0008) -[2023-10-15 18:39:59,778][52866] Updated weights for policy 1, policy_version 97570 (0.0009) -[2023-10-15 18:40:00,111][52833] Updated weights for policy 0, policy_version 97330 (0.0009) -[2023-10-15 18:40:00,131][52866] Updated weights for policy 1, policy_version 97580 (0.0008) -[2023-10-15 18:40:00,472][52833] Updated weights for policy 0, policy_version 97340 (0.0007) -[2023-10-15 18:40:00,498][52866] Updated weights for policy 1, policy_version 97590 (0.0007) -[2023-10-15 18:40:00,859][52866] Updated weights for policy 1, policy_version 97600 (0.0008) -[2023-10-15 18:40:03,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 199622656. Throughput: 0: 1784.9, 1: 1801.6. Samples: 49910550. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:03,443][51532] Avg episode reward: [(0, '93.290'), (1, '69.660')] -[2023-10-15 18:40:04,226][52833] Updated weights for policy 0, policy_version 97350 (0.0009) -[2023-10-15 18:40:04,585][52833] Updated weights for policy 0, policy_version 97360 (0.0008) -[2023-10-15 18:40:04,755][52866] Updated weights for policy 1, policy_version 97610 (0.0007) -[2023-10-15 18:40:04,955][52833] Updated weights for policy 0, policy_version 97370 (0.0009) -[2023-10-15 18:40:05,129][52866] Updated weights for policy 1, policy_version 97620 (0.0007) -[2023-10-15 18:40:05,504][52866] Updated weights for policy 1, policy_version 97630 (0.0008) -[2023-10-15 18:40:08,441][51532] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 199688192. Throughput: 0: 1790.1, 1: 1793.8. Samples: 49932868. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:08,442][51532] Avg episode reward: [(0, '92.990'), (1, '69.630')] -[2023-10-15 18:40:08,535][52833] Updated weights for policy 0, policy_version 97380 (0.0009) -[2023-10-15 18:40:08,910][52833] Updated weights for policy 0, policy_version 97390 (0.0008) -[2023-10-15 18:40:09,264][52833] Updated weights for policy 0, policy_version 97400 (0.0007) -[2023-10-15 18:40:09,331][52866] Updated weights for policy 1, policy_version 97640 (0.0010) -[2023-10-15 18:40:09,704][52866] Updated weights for policy 1, policy_version 97650 (0.0008) -[2023-10-15 18:40:10,066][52866] Updated weights for policy 1, policy_version 97660 (0.0007) -[2023-10-15 18:40:12,981][52833] Updated weights for policy 0, policy_version 97410 (0.0007) -[2023-10-15 18:40:13,347][52833] Updated weights for policy 0, policy_version 97420 (0.0009) -[2023-10-15 18:40:13,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 199753728. Throughput: 0: 1806.1, 1: 1796.9. Samples: 49955642. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:13,441][51532] Avg episode reward: [(0, '91.000'), (1, '69.750')] -[2023-10-15 18:40:13,662][52866] Updated weights for policy 1, policy_version 97670 (0.0009) -[2023-10-15 18:40:13,712][52833] Updated weights for policy 0, policy_version 97430 (0.0008) -[2023-10-15 18:40:14,019][52866] Updated weights for policy 1, policy_version 97680 (0.0008) -[2023-10-15 18:40:14,077][52833] Updated weights for policy 0, policy_version 97440 (0.0008) -[2023-10-15 18:40:14,377][52866] Updated weights for policy 1, policy_version 97690 (0.0007) -[2023-10-15 18:40:17,764][52833] Updated weights for policy 0, policy_version 97450 (0.0010) -[2023-10-15 18:40:18,121][52866] Updated weights for policy 1, policy_version 97700 (0.0009) -[2023-10-15 18:40:18,131][52833] Updated weights for policy 0, policy_version 97460 (0.0008) -[2023-10-15 18:40:18,441][51532] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 199819264. Throughput: 0: 1789.0, 1: 1800.5. Samples: 49965536. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:18,441][51532] Avg episode reward: [(0, '93.220'), (1, '68.510')] -[2023-10-15 18:40:18,496][52866] Updated weights for policy 1, policy_version 97710 (0.0008) -[2023-10-15 18:40:18,500][52833] Updated weights for policy 0, policy_version 97470 (0.0007) -[2023-10-15 18:40:18,852][52866] Updated weights for policy 1, policy_version 97720 (0.0007) -[2023-10-15 18:40:22,441][52833] Updated weights for policy 0, policy_version 97480 (0.0009) -[2023-10-15 18:40:22,627][52866] Updated weights for policy 1, policy_version 97730 (0.0007) -[2023-10-15 18:40:22,809][52833] Updated weights for policy 0, policy_version 97490 (0.0007) -[2023-10-15 18:40:22,995][52866] Updated weights for policy 1, policy_version 97740 (0.0009) -[2023-10-15 18:40:23,185][52833] Updated weights for policy 0, policy_version 97500 (0.0007) -[2023-10-15 18:40:23,350][52866] Updated weights for policy 1, policy_version 97750 (0.0009) -[2023-10-15 18:40:23,441][51532] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 199917568. Throughput: 0: 1803.3, 1: 1793.5. Samples: 49988032. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:23,442][51532] Avg episode reward: [(0, '92.860'), (1, '63.160')] -[2023-10-15 18:40:23,714][52866] Updated weights for policy 1, policy_version 97760 (0.0008) -[2023-10-15 18:40:26,979][52833] Updated weights for policy 0, policy_version 97510 (0.0007) -[2023-10-15 18:40:27,354][52833] Updated weights for policy 0, policy_version 97520 (0.0007) -[2023-10-15 18:40:27,496][52866] Updated weights for policy 1, policy_version 97770 (0.0009) -[2023-10-15 18:40:27,718][52833] Updated weights for policy 0, policy_version 97530 (0.0007) -[2023-10-15 18:40:27,865][52866] Updated weights for policy 1, policy_version 97780 (0.0009) -[2023-10-15 18:40:28,239][52866] Updated weights for policy 1, policy_version 97790 (0.0010) -[2023-10-15 18:40:28,441][51532] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 200015872. Throughput: 0: 1805.9, 1: 1799.5. Samples: 50008062. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:28,442][51532] Avg episode reward: [(0, '86.860'), (1, '62.640')] -[2023-10-15 18:40:31,411][52833] Updated weights for policy 0, policy_version 97540 (0.0007) -[2023-10-15 18:40:31,784][52833] Updated weights for policy 0, policy_version 97550 (0.0009) -[2023-10-15 18:40:32,105][52866] Updated weights for policy 1, policy_version 97800 (0.0009) -[2023-10-15 18:40:32,155][52833] Updated weights for policy 0, policy_version 97560 (0.0009) -[2023-10-15 18:40:32,460][52866] Updated weights for policy 1, policy_version 97810 (0.0008) -[2023-10-15 18:40:32,824][52866] Updated weights for policy 1, policy_version 97820 (0.0008) -[2023-10-15 18:40:33,441][51532] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 200081408. Throughput: 0: 1801.8, 1: 1790.3. Samples: 50020084. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:33,442][51532] Avg episode reward: [(0, '84.350'), (1, '64.390')] -[2023-10-15 18:40:35,916][52833] Updated weights for policy 0, policy_version 97570 (0.0009) -[2023-10-15 18:40:36,284][52833] Updated weights for policy 0, policy_version 97580 (0.0008) -[2023-10-15 18:40:36,607][52866] Updated weights for policy 1, policy_version 97830 (0.0009) -[2023-10-15 18:40:36,651][52833] Updated weights for policy 0, policy_version 97590 (0.0009) -[2023-10-15 18:40:36,971][52866] Updated weights for policy 1, policy_version 97840 (0.0008) -[2023-10-15 18:40:37,014][52833] Updated weights for policy 0, policy_version 97600 (0.0008) -[2023-10-15 18:40:37,336][52866] Updated weights for policy 1, policy_version 97850 (0.0007) -[2023-10-15 18:40:38,441][51532] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 200146944. Throughput: 0: 1804.1, 1: 1805.3. Samples: 50040192. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:38,442][51532] Avg episode reward: [(0, '81.380'), (1, '66.180')] -[2023-10-15 18:40:40,793][52833] Updated weights for policy 0, policy_version 97610 (0.0009) -[2023-10-15 18:40:41,080][52866] Updated weights for policy 1, policy_version 97860 (0.0011) -[2023-10-15 18:40:41,154][52833] Updated weights for policy 0, policy_version 97620 (0.0009) -[2023-10-15 18:40:41,434][52866] Updated weights for policy 1, policy_version 97870 (0.0008) -[2023-10-15 18:40:41,536][52833] Updated weights for policy 0, policy_version 97630 (0.0008) -[2023-10-15 18:40:41,813][52866] Updated weights for policy 1, policy_version 97880 (0.0009) -[2023-10-15 18:40:43,441][51532] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 200212480. Throughput: 0: 1797.5, 1: 1784.3. Samples: 50061682. Policy #0 lag: (min: 1.0, avg: 10.7, max: 33.0) -[2023-10-15 18:40:43,442][51532] Avg episode reward: [(0, '80.250'), (1, '68.170')] -[2023-10-15 18:40:45,277][52833] Updated weights for policy 0, policy_version 97640 (0.0008) -[2023-10-15 18:40:45,497][52866] Updated weights for policy 1, policy_version 97890 (0.0009) -[2023-10-15 18:40:45,653][52833] Updated weights for policy 0, policy_version 97650 (0.0009) -[2023-10-15 18:40:45,859][52866] Updated weights for policy 1, policy_version 97900 (0.0009) -[2023-10-15 18:40:46,032][52833] Updated weights for policy 0, policy_version 97660 (0.0009) -[2023-10-15 18:40:46,222][52866] Updated weights for policy 1, policy_version 97910 (0.0007) -[2023-10-15 18:40:46,589][52878] Stopping RolloutWorker_w10... -[2023-10-15 18:40:46,589][52871] Stopping RolloutWorker_w3... -[2023-10-15 18:40:46,589][52875] Stopping RolloutWorker_w6... -[2023-10-15 18:40:46,589][52881] Stopping RolloutWorker_w13... -[2023-10-15 18:40:46,590][52878] Loop rollout_proc10_evt_loop terminating... -[2023-10-15 18:40:46,589][52410] Stopping Batcher_0... -[2023-10-15 18:40:46,589][52874] Stopping RolloutWorker_w5... -[2023-10-15 18:40:46,589][51532] Component RolloutWorker_w10 stopped! -[2023-10-15 18:40:46,590][52871] Loop rollout_proc3_evt_loop terminating... -[2023-10-15 18:40:46,590][52875] Loop rollout_proc6_evt_loop terminating... -[2023-10-15 18:40:46,590][52877] Stopping RolloutWorker_w9... -[2023-10-15 18:40:46,590][52881] Loop rollout_proc13_evt_loop terminating... -[2023-10-15 18:40:46,589][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000097920_100270080.pth... -[2023-10-15 18:40:46,590][52874] Loop rollout_proc5_evt_loop terminating... -[2023-10-15 18:40:46,590][52877] Loop rollout_proc9_evt_loop terminating... -[2023-10-15 18:40:46,590][51532] Component RolloutWorker_w3 stopped! -[2023-10-15 18:40:46,590][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... -[2023-10-15 18:40:46,590][52882] Stopping RolloutWorker_w11... -[2023-10-15 18:40:46,590][51532] Component Batcher_0 stopped! -[2023-10-15 18:40:46,591][52882] Loop rollout_proc11_evt_loop terminating... -[2023-10-15 18:40:46,591][52872] Stopping RolloutWorker_w4... -[2023-10-15 18:40:46,591][51532] Component RolloutWorker_w6 stopped! -[2023-10-15 18:40:46,591][52873] Stopping RolloutWorker_w2... -[2023-10-15 18:40:46,591][52866] Updated weights for policy 1, policy_version 97920 (0.0010) -[2023-10-15 18:40:46,591][52872] Loop rollout_proc4_evt_loop terminating... -[2023-10-15 18:40:46,591][51532] Component RolloutWorker_w13 stopped! -[2023-10-15 18:40:46,591][52873] Loop rollout_proc2_evt_loop terminating... -[2023-10-15 18:40:46,591][51532] Component RolloutWorker_w5 stopped! -[2023-10-15 18:40:46,592][51532] Component RolloutWorker_w9 stopped! -[2023-10-15 18:40:46,592][51532] Component Batcher_1 stopped! -[2023-10-15 18:40:46,592][52870] Stopping RolloutWorker_w1... -[2023-10-15 18:40:46,592][51532] Component RolloutWorker_w11 stopped! -[2023-10-15 18:40:46,592][53658] Stopping RolloutWorker_w15... -[2023-10-15 18:40:46,592][51532] Component RolloutWorker_w4 stopped! -[2023-10-15 18:40:46,593][52870] Loop rollout_proc1_evt_loop terminating... -[2023-10-15 18:40:46,592][52879] Stopping RolloutWorker_w7... -[2023-10-15 18:40:46,593][51532] Component RolloutWorker_w2 stopped! -[2023-10-15 18:40:46,593][52869] Stopping RolloutWorker_w0... -[2023-10-15 18:40:46,593][53658] Loop rollout_proc15_evt_loop terminating... -[2023-10-15 18:40:46,593][52879] Loop rollout_proc7_evt_loop terminating... -[2023-10-15 18:40:46,593][51532] Component RolloutWorker_w1 stopped! -[2023-10-15 18:40:46,593][52880] Stopping RolloutWorker_w12... -[2023-10-15 18:40:46,593][52869] Loop rollout_proc0_evt_loop terminating... -[2023-10-15 18:40:46,593][52880] Loop rollout_proc12_evt_loop terminating... -[2023-10-15 18:40:46,593][51532] Component RolloutWorker_w15 stopped! -[2023-10-15 18:40:46,594][51532] Component RolloutWorker_w7 stopped! -[2023-10-15 18:40:46,594][52876] Stopping RolloutWorker_w8... -[2023-10-15 18:40:46,594][53503] Stopping RolloutWorker_w14... -[2023-10-15 18:40:46,594][51532] Component RolloutWorker_w0 stopped! -[2023-10-15 18:40:46,594][52876] Loop rollout_proc8_evt_loop terminating... -[2023-10-15 18:40:46,594][51532] Component RolloutWorker_w12 stopped! -[2023-10-15 18:40:46,594][53503] Loop rollout_proc14_evt_loop terminating... -[2023-10-15 18:40:46,595][51532] Component RolloutWorker_w8 stopped! -[2023-10-15 18:40:46,595][51532] Component RolloutWorker_w14 stopped! -[2023-10-15 18:40:46,590][52518] Stopping Batcher_1... -[2023-10-15 18:40:46,590][52410] Loop batcher_evt_loop terminating... -[2023-10-15 18:40:46,617][52833] Weights refcount: 2 0 -[2023-10-15 18:40:46,617][52866] Weights refcount: 2 0 -[2023-10-15 18:40:46,619][52833] Stopping InferenceWorker_p0-w0... -[2023-10-15 18:40:46,619][52866] Stopping InferenceWorker_p1-w0... -[2023-10-15 18:40:46,619][51532] Component InferenceWorker_p0-w0 stopped! -[2023-10-15 18:40:46,619][52833] Loop inference_proc0-0_evt_loop terminating... -[2023-10-15 18:40:46,619][52866] Loop inference_proc1-0_evt_loop terminating... -[2023-10-15 18:40:46,620][51532] Component InferenceWorker_p1-w0 stopped! -[2023-10-15 18:40:46,612][52518] Loop batcher_evt_loop terminating... -[2023-10-15 18:40:46,640][52410] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000096384_98697216.pth -[2023-10-15 18:40:46,640][52518] Removing ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000096640_98959360.pth -[2023-10-15 18:40:46,646][52410] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... -[2023-10-15 18:40:46,646][52518] Saving ./train_atari/atari_spaceinvaders_APPO/checkpoint_p1/checkpoint_000097920_100270080.pth... -[2023-10-15 18:40:46,705][52410] Stopping LearnerWorker_p0... -[2023-10-15 18:40:46,706][52410] Loop learner_proc0_evt_loop terminating... -[2023-10-15 18:40:46,706][51532] Component LearnerWorker_p0 stopped! -[2023-10-15 18:40:46,706][52518] Stopping LearnerWorker_p1... -[2023-10-15 18:40:46,707][51532] Component LearnerWorker_p1 stopped! -[2023-10-15 18:40:46,707][52518] Loop learner_proc1_evt_loop terminating... -[2023-10-15 18:40:46,707][51532] Waiting for process learner_proc0 to stop... -[2023-10-15 18:40:47,599][51532] Waiting for process learner_proc1 to stop... -[2023-10-15 18:40:47,629][51532] Waiting for process inference_proc0-0 to join... -[2023-10-15 18:40:47,629][51532] Waiting for process inference_proc1-0 to join... -[2023-10-15 18:40:47,630][51532] Waiting for process rollout_proc0 to join... -[2023-10-15 18:40:47,631][51532] Waiting for process rollout_proc1 to join... -[2023-10-15 18:40:47,632][51532] Waiting for process rollout_proc2 to join... -[2023-10-15 18:40:47,632][51532] Waiting for process rollout_proc3 to join... -[2023-10-15 18:40:47,633][51532] Waiting for process rollout_proc4 to join... -[2023-10-15 18:40:47,633][51532] Waiting for process rollout_proc5 to join... -[2023-10-15 18:40:47,634][51532] Waiting for process rollout_proc6 to join... -[2023-10-15 18:40:47,635][51532] Waiting for process rollout_proc7 to join... -[2023-10-15 18:40:47,635][51532] Waiting for process rollout_proc8 to join... -[2023-10-15 18:40:47,636][51532] Waiting for process rollout_proc9 to join... -[2023-10-15 18:40:47,637][51532] Waiting for process rollout_proc10 to join... -[2023-10-15 18:40:47,638][51532] Waiting for process rollout_proc11 to join... -[2023-10-15 18:40:47,638][51532] Waiting for process rollout_proc12 to join... -[2023-10-15 18:40:47,639][51532] Waiting for process rollout_proc13 to join... -[2023-10-15 18:40:47,640][51532] Waiting for process rollout_proc14 to join... -[2023-10-15 18:40:47,640][51532] Waiting for process rollout_proc15 to join... -[2023-10-15 18:40:47,641][51532] Batcher 0 profile tree view: -batching: 171.7417, releasing_batches: 0.0945 -[2023-10-15 18:40:47,641][51532] Batcher 1 profile tree view: -batching: 170.2844, releasing_batches: 0.0907 -[2023-10-15 18:40:47,642][51532] InferenceWorker_p0-w0 profile tree view: -wait_policy: 0.0001 - wait_policy_total: 1942.3324 -update_model: 198.5397 - weight_update: 0.0009 -one_step: 0.0024 - handle_policy_step: 11155.2173 - deserialize: 62.4104, stack: 190.2160, obs_to_device_normalize: 2483.0861, forward: 5049.8731, prepare_outputs: 2427.6340, send_messages: 459.1723 -[2023-10-15 18:40:47,642][51532] InferenceWorker_p1-w0 profile tree view: -wait_policy: 0.0001 - wait_policy_total: 1895.8374 -update_model: 206.6028 - weight_update: 0.0010 -one_step: 0.0024 - handle_policy_step: 11201.2031 - deserialize: 63.3332, stack: 194.1396, obs_to_device_normalize: 2508.8051, forward: 5059.1440, prepare_outputs: 2439.8848, send_messages: 461.5081 -[2023-10-15 18:40:47,642][51532] Learner 0 profile tree view: -misc: 0.0207, prepare_batch: 269.5069 -train: 3621.2179 - epoch_init: 0.1934, minibatch_init: 12.9289, losses_postprocess: 890.4475, kl_divergence: 31.4494, update: 387.3310, after_optimizer: 2115.4655 - calculate_losses: 166.3816 - losses_init: 0.4056, forward_head: 55.5511, bptt_initial: 1.4002, bptt: 1.9765, tail: 38.0968, advantages_returns: 11.3436, losses: 44.0924 -[2023-10-15 18:40:47,642][51532] Learner 1 profile tree view: -misc: 0.0185, prepare_batch: 269.1968 -train: 3604.0339 - epoch_init: 0.1876, minibatch_init: 13.0160, losses_postprocess: 889.8636, kl_divergence: 31.0945, update: 384.9851, after_optimizer: 2099.4010 - calculate_losses: 168.4524 - losses_init: 0.3766, forward_head: 59.0517, bptt_initial: 1.4405, bptt: 1.9506, tail: 37.5311, advantages_returns: 10.9481, losses: 43.6357 -[2023-10-15 18:40:47,643][51532] RolloutWorker_w0 profile tree view: -wait_for_trajectories: 1.2240, enqueue_policy_requests: 405.9331, process_policy_outputs: 191.4867, env_step: 6575.9914, finalize_trajectories: 3.5020, complete_rollouts: 2.9397 -post_env_step: 381.5736 - process_env_step: 85.9592 -[2023-10-15 18:40:47,643][51532] RolloutWorker_w15 profile tree view: -wait_for_trajectories: 1.2264, enqueue_policy_requests: 408.1219, process_policy_outputs: 189.6638, env_step: 6596.0681, finalize_trajectories: 3.4623, complete_rollouts: 2.8993 -post_env_step: 375.2059 - process_env_step: 83.1213 -[2023-10-15 18:40:47,643][51532] Loop Runner_EvtLoop terminating... -[2023-10-15 18:40:47,644][51532] Runner profile tree view: -main_loop: 13979.6057 -[2023-10-15 18:40:47,644][51532] Collected {0: 100007936, 1: 100270080}, FPS: 14326.4 +version https://git-lfs.github.com/spec/v1 +oid sha256:5914724ae73643c71b319dc5b1c195a0c958accf0f6864b43e0c519e156c6946 +size 48532816